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I.  INTRODUCTION 


A.  MOTIVATION  FOR  VISION  DEVELOPMENT 

Image  understanding  has  been  a  difficult  subject  in  the  fields  of  artificial  intelligence  and  robotics  for 
nearly  twenty  years.  The  prospect  of  developing  a  novel  and  revolutionary  approach  to  image  processing  and 
image  understanding  in  the  course  of  writing  a  master’s  thesis  seems  unlikely.  Instead,  as  with  most  all  vision 
systems  developed,  this  approach  will  be  directed  towards  developing  a  visual  navigation  system  based  upon 
the  specific  needs  of  the  robot  and  tailoring  the  application  to  the  domain  of  its  operating  environment. 
Certain  assumptions  will  be  made  to  simplify  the  problem.  These  assumptions  are: 

-  the  robot  will  operate  in  an  indoor,  orthogonal  environment, 

-  the  floor  will  be  a  flat  surface, 

-  the  floorplan  of  the  environment  will  be  known, 

-  the  robot  will  always  be  in  an  upright  position, 

-  and  approximate  position  information  will  be  accessible. 

The  image  understanding  techniques  to  be  described  have  been  developed  utilizing  a  charge-coupled 
device  (CCD)  television  camera,  a  graphics  workstation,  and  a  video  frame  grabber.  The  CCD  television 
camera  (JVC  model  TK870U)  is  capable  of  providing  digitized  RGB  formatted  color  information  (red,  green, 
and  blue  components)  to  an  image  processing  system.  The  graphics  workstation  is  a  SiliconGraphics  Personal 
Iris  and  the  video  frame  grabbing  board  is  SiliconGraphics’  VideoFramer.  The  SiliconGraphics  graphics 
library  supports  basic  display  functions  in  the  C  language.  The  vision  based  routines  described  within  this 
thesis  are  programmed  in  the  ANSI  C  language. 

B.  A  UTONOMOUS  MOBILE  ROBOT  SYSTEM  Y AM ABICO-1 1 

Yamabico-11  (Figure  1.1)  is  an  autonomous  mobile  robot  that  has  been  the  focus  of  many  students' 
theses  at  the  Naval  Postgraduate  School.  Work  in  the  fields  of  motion  control,  safe  path  planning,  and  sonar- 
based  navigation  are  continually  ongoing  to  upgrade  the  robot’s  capabilities.  Yamabico’s  platform  is  an 
aluminum  cart  that  stands  36  inches  tall,  with  two  main  wheels  for  motion  and  steering  and  four  smaller 
wheels  for  support  Its  maneuvering  capabilities  resemble  that  of  a  tracked  vehicle  (it  can  pivot  in  place). 
Power  is  provided  from  two  12-volt  wheelchair  batteries. 

Process  control  is  conducted  by  one  Motorola  68020  32-bit  microprocessor  with  an  accompanying 
Motorola  6888 1  floating  point  coprocessor.  The  processor  has  1Mbyte  of  onboard  RAM  and  runs  at  16MHz. 
Control  information  is  passed  over  a  VME  bus  which  also  carries  a  dual  axis  controller  board  and  two  4- 
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channel  serial  communication  boards.  The  dual  axis  controller  interfaces  with  driving  and  braking  motors. 
The  serial  channels  interface  with  an  onboard  terminal  and  a  modem  to  a  host  computer. 

Its  only  sensors,  twelve  ultrasonic  sonars  that  operate  at  40kHz,  are  mounted  about  the  lower  base  and 
are  controlled  by  three  Motorola  6809-based  microprocessors.  The  Sonar  Control  Language  [SHE91]  and  the 
Mobile  Motion  Language  (MML)  [KAN89a]  provide  easy  sonar  and  motion  control  features  to  the 
programmer  and  are  downloaded  to  Yamabico  via  an  RS232  serial  port  from  the  host  computer,  a  Sun 
Microsystems  3/160.  The  system  is  composed  of  a  kernel  and  a  user-program.  The  kernel  (about  100  kbytes) 
only  needs  to  be  downloaded  once.  The  user-program,  userO,  specifies  Yamabico’s  tasks.  Once  the  object 
code  has  been  downloaded  and  the  serial  link  disconnected,  all  computation  is  performed  autonomously  and 
is  executed  via  a  keyboard  mounted  on  top  of  Yamabico. 

Presently,  Yamabico  can  navigate  its  way  through  a  “known”  corridor  by  utilizing  ultra-sonic  range 
information  to  update  its  position  which  is  maintained  by  wheel  motion  dead-reckoning.  Yamabico’s  ultra¬ 
sonic  sensors  are  effective  in  ranges  from  9.6cm  to  400cm  and  are  accurate  to  18.95mm  at  a  range  of  500mm. 
The  range  information  can  directly  update  the  current  position  in  a  known  environment  (such  as  our  testing 
grounds,  the  fifth  floor  of  Spanage!  Hall  at  the  Naval  Postgraduate  School).  Prior  to  the  work  outlined  in  this 
thesis.  Yamabico  had  no  image  based  sensor  capabilities  at  all.  The  purpose  of  this  thesis  is  to  develop  and 
implement  a  vision  based  sensor  system  for  Yamabico- 11  suitable  for  navigation  and  exploration  of  its 
environment. 


Figure  1.1  The  autonomous  mobile  robot  Yamabico-1 1 . 
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C.  LITERATURE  SURVEY 


Image  understanding  has  been  a  complex  topic  for  computer  scientists  lor  almost  two  decades.  The 
ideas  and  algorithms  utilized  in  this  work  stem  from  the  image  processing,  linear  fitting,  pattern  matching, 
and  pose  determination  methods  that  have  been  found  to  be  successful  in  past  applications.  Two  good  sources 
for  the  basic  ideas  necessary  in  image  understanding  are  by  Ballard  [BAL821  and  Nevada  [NEV82],  These 
are  both  textbooks  that  outline  the  more  commonly  used  methods  and  the  background  fundamentals.  They 
cover  a  wide  range  of  topics  including  low  level  image  processing,  graph-theory  for  pattern  matching,  and 
model-base  object  recognition. 

A  paper  by  Marr  [M  AR79]  outlines  requirements  for  model-based  image  understanding  and  provides 
insight  for  construction  of  suitable  models,  similar  to  his  “2D+,”  model  for  vision  applications.  Stein’s  thesis 
[STE92]  describes  the  environment  model  developed  for  the  vision  navigation  method  developed  in  this 
thesis.  Two  papers  by  Nanay ama.  |KAN90a]  and  [KAN90b|,  were  consulted  for  familiarization  with  the 
robot  Yamabico's  motion  contiol  capabilities  (with  the  MML  language)  and  the  application  of  linear  fitting 
of  sonar  point  data  by  least  squares  lor  ein  ironment  feature  mapping.  Sherley  s  thesis  [SHE91  ]  provides  the 
details  on  Yamabico's  sonar  capabilities. 

A  lot  of  recently  published  research  conducted  at  the  University  of  Massachusetts  at  Amherst  provided 
motivation  to  develop  Lister  and  simpler  image  edge  extraction  and  visual  navigation  routines.  Papers  from 
this  group  bv  Beveridge  ,BE\90;.  Kumar  |KUM89|,  and  Fennema  |FEN90j  describe  methods  for  pattern 
matching,  pose  determination,  and  their  experimental  results.  Edge  extraction  was  conducted  via  the  Hough 
transform  and  pattern  mat*,  lung  performed  by  random  combinations  of  image  to  model  line  matches. 

Pose  determination  methods  described  by  Haralick  [HARS9J  and  Liu  1L1U90)  are  algorithms  based 
upon  the  general  trigonometric  principles  of  photogrammetry.  A  photogrammeiry  textbook,  such  as  Moffit’s 
[MOF80J,  proved  to  be  a  good  source  to  explain  these  fundamentals.  During  development  of  this  algorithm, 
a  means  for  pose  verification  was  found  to  be  necessary.  Methods  tested  by  Heller  [HEL89]  show  the  results 
of  verifying  all  image  lines  w  uh  model  lines  and  the  results  of  verifying  model  lines  with  edge-related  pixel 
data.  A  basic  computer  graphics  textbook  by  Hall  [HAL89|  was  also  consulted  for  checking  video  source 
formats  and  computer  image  data  formats. 


D.  THESIS  ORGANIZATION 


The  following  Chapters  outline  the  development  of  the  algorithm  applied  towards  a  simple  visual 
navigation  scheme  for  an  autonomous  mobile  robot  in  a  known  orthogonal  environment.  The  objectives  are 
stated  in  Chapter  II.  Chapter  in  presents  the  general  method  of  feature  extraction  based  upon  determining  the 
edges  depicted  in  an  image.  Chapter  IV  discuses  the  method  in  which  the  image’s  pixels  are  grouped  into 
two-dimensional  regions  describing  the  edge  • ">n  of  straight  line  segments  from  two-dimensional 
areas  based  upon  least-squares  fitting  of  datapoinis  is ,  "banter  V.  The  straight  edge  extraction 

algorithm  is  then  outlined  in  Chapter  VI.  The  algorithm’s  re-u...  .  Tects  of  varying  parameters  within 

the  algorithm  are  shown  in  Chapter  VII.  Chapter  VUI  discusses  imp.  "  factors  such  as  image  data 

structures  and  the  environment  model. 

Chapter  IX  discusses  the  method  for  pattern  matching  of  extracted  edges  frooi .  vitta  modelled 

environment  features.  Chapter  X  describes  pose  (the  vehicle’s  position  and  onentation  within  the 
environment)  determination  based  upon  disparities  between  the  image  and  the  expected  view  of  the  modelled 
environment  from  an  assumed  position.  Pose  determination  results  are  shown  in  Chapter  XI.  Conclusions  and 
subsequent  topics  for  this  visual  navigation  system  are  presented  in  Chapter  XU. 

Appendix  A  specifies  the  atan2  function  used  for  determination  of  a  vector  orientation  and  it  contains 
the  data  types  header  files  used.  Appendix  B  contains  the  edge  extraction  implementations  findedge  and 
fastedge  and  all  the  required  header  files  for  compilation.  Appendix  C  contains  the  implementation  vertmatch 
which  extracts  vertical  image  lines,  matches  them  with  the  model  lines  from  an  expected  two-dimensional 
view  of  the  environment,  and  determines  the  robot’s  correct  position  and  orientation.  Appendix  D  contains  a 
user’s  guide  for  the  compilation  and  operation  of  the  edge  extraction  and  pose  determination  programs. 
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H.  OBJECTIVES 


There  are  two  primary  objectives  for  the  development  of  a  vision  understanding  system  for  the 
Yamabico-1 1  robot.  They  are  similar  to  the  objectives  for  any  autonomous  mobile  vehicle. 

1.  Update  correct  vehicle  position  and  orientation  for  precise  navigation. 

2.  Recognize  specific  objects  using  a  prior  knowledge  based  upon  the  task  requirements  and  the 
operational  environment. 

A.  POSITION  /  ORIENTATION  IDENTIFICATION  IN  A  KNOWN  WORLD 

Determining  accurate  position  and  orientation  autonomously  is  a  crucial  requirement  for  the  successful 
and  safe  navigation  of  any  mobile  robot.  This  task  is  also  known  as  pose  determination  and  is  used  to  update 
the  robot's  positional  information  that  is  maintained  by  wheel  motion  dead-reckoning.  Small  errors  from 
dead-reckoning  arise  quickly  and  are  compounded  by  turns  and  acceleration.  Continuous  periodic  pose 
determination  is  fundamental  for  precise  dead-reckoning  based  navigation.  Chapter  X  outlines  a  simple 
position-correction  algorithm  via  pattern-matching  of  the  extracted  features  from  a  two-dimensional  video 
image  and  two-dimensional  view  of  mapped  features  of  the  three-dimensional  environment  model 
maintained  by  the  robot.  This  algorithm  is  based  upon  previous  works  [HAR89],  [MAR79],  fMOF80], 
[KUM89],  [FEN90],  and  fIJU90].  whose  roots  stem  from  shipboard  navigation  and  aerial  photogrammetry. 

B.  OBJECT  RECOGNITION  IN  AN  UNKNOWN  OR  PARTIALLY  KNOWN  WORLD 

The  second  objective  is  the  recognition  of  specific  objects  via  three-dimensional  models.  This  is  a 
much  more  complex  task  than  the  first  due  to  the  uncertainty  of  individual  objects  presence  and  location.  The 
first  task  of  pose  determination  is  a  subset  of  this  task.  The  operating  environment  is  modelled  as  a  three- 
dimensional  object  and  determining  the  camera’s  pose  is  only  determination  of  viewing  aspect  of  the  object. 
Object  recognition  covers  many  other  topics  including  mapping  an  unknown  environment,  searching  for  a 
specific  goal,  classifying  encountered  objects,  and  avoiding  unknown  obstacles.  Some  aspects  of  object 
recognition  are  detailed  in  [BAL82],  [MAR79],  [NEV82],  and  many  others.  A  central  theme  of  all  vision- 
based  object  recognition  applications  is  modelling  and  searching  for  only  the  objects  that  are  expected  to  be 
encountered  and  classifying  unknown  items  by  their  geometrical  properties  This  objective  is  not  approached 
within  this  paper,  but  will  hopefully  be  accomplished  for  Yamabico  by  a  successor 
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IE.  METHOD 


A.  ARCHITECTURE  FOR  IMAGE  BASED  NAVIGATION 

Extracting  desired  features  from  an  image  is  a  crucial  process  in  any  image  understanding 
implementation.  Some  methods  focus  on  determining  areas  of  common  light  intensity,  color,  or  texture. 
These  methods  often  include  segmentation  of  the  image  by  region  growing  and  are  generally  suitable  for 
images  with  small  areas  of  pixels  with  common  attributes  (e.g.  a  picture  of  a  forest).  Other  methods  search 
for  the  edges  between  contrasting  bordering  areas  of  pixels  with  similar  attributes.  Likewise,  these  methods 
are  suitable  for  images  with  large  areas  of  common  light  intensity,  color,  or  texture  (e.g.  a  picture  that 
included  bare  walls).  This  method  will  be  of  the  latter  category;  extracting  the  edge  features  of  an  image.  This 
method  was  chosen  for  recognizing  the  straight  edges  of  orthogonal  o  whose  surfaces  are  often  uniform 

in  color  and  texture.  The  images  that  the  edge  extraction  methods  win  lred  to,  will  be  pictures  of  the 
interiors  of  offices  and  hallways.  Pictures  where  straight  line  segments  would  b;.  common  and  suitable  for 
describing  the  objects  in  the  image. 

The  goal  of  the  image  processing  part  of  the  vision  system  will  be  to  find  straight  line  segments  in  the 
edges  of  boundaries  between  areas  of  similar  light  intensity  in  an  image.  The  straight  line  segments  should 
be  a  sufficiently  simple  data  structure  for  use  in  follow-on  image  understanding  implementations.  The 
specific  problem  of  image  understanding  for  navigation  of  Yamabico  in  a  known,  orthogonal  environment 
will  be  split  into  the  following  subproblems: 

1 .  Image  Processing  -  extracting  desired  features  (straight  line  edges)  from  the  input  image. 

2.  Pattern  Matching  -  correlating  extracted  features  with  known  features  described  in  a  three- 
dimensional  model  of  the  environment. 

3.  Pose  Determination  -  calculating  the  position  and  orientation  of  the  robot  within  its  modelled 
operating  environment. 

Since  Yamabico  has  no  vision  based  sensor  capabiir:  ;.v.  lutiowing  features  were  identified  for  the 
implementation  of  a  vision  understanding  system  (Figure  3.1) 

-  a  three-dimensional  model  representation  of  the  operating  environment, 

-  camera  image  processing  functions, 

-  image  feature  extracting  functions, 

-  pattern  matching  routines, 

-  and  visual-based  position  correction  methods. 
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Figure  3.1.  The  general  outline  for  image  understanding  on  Yamabico-11. 


In  order  to  accomplish  the  image  processing  task  of  finding  straight  line  segments  in  an  image,  the 
borders  (edges)  between  areas  will  be  identified,  segmented,  and  then  simplified  in  terms  of  straight  line 
segments  ( Figure  3.2).  The  identification  of  the  edges  in  an  image  (Figure  3.3)  is  best  described  by  a  gradient 
image  (Figure  3.4).  Two  important  steps  to  follow  are: 

1 .  grouping  of  pixels  into  contiguous  regions  of  separate  edges,  and 

2.  linear  fitting  of  the  edges. 
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Figure  3.2.  Outline  of  method  for  edge  extraction  of  an  image.  Edges  will  be  represented 
as  line  segments. 


Thus,  once  the  gradient  image  is  made,  pixels  that  define  the  edges  (the  black  pixels  of  Figure  3.4)  must 
be  grouped  and  then  fitted  to  a  straight  line  segment  by  least  squares  fit  to  the  major  axis  of  the  edge  region. 
Determining  when  to  start  and  stop  constructing  separate  edges  is  a  crucial  subject.  Checking  the  validity  of 
the  line  segments  will  be  the  final  requirement  of  the  image  processing  task. 
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Figure  3.4  The  gradient  image  lor  ihe  chair  in  Figure  3.3.  Edges  between  regions  of 
common  light  intensities  are  depicted  by  black  pixels. 
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Figure  5.5.  The  extracted  line  segments  based  upon  the  grouping  of  similar  edge-related  pixels. 

The  pattern  matching  process  will  require  two  inputs.  These  will  be  the  linear  features  from  the  three- 
dimensional  model  mapped  onto  a  two-dimensional  viewing  plane  and  edges  extracted  from  the  image 
processing  system.  The  three-dimensional  model  is  a  two-dimensional  lloorplan  plus  vertical  components  for 
height  measurement.  This  model  will  be  used  to  construct  a  two-dimensional  “wire-frame”  view  of  the 
environment  from  any  given  pose  for  the  robot’s  camera.  This  view  will  be  used  for  what  the  robot  will  expect 
to  see  from  its  present  location  and  orientation.  The  three-dimensional  modelling  system  for  Yamabico  is 
being  pursued  concurrently  w  ith  this  work  by  Li.  Jim  Stein  for  his  master’s  thesis  [STE92]. 

The  image  processing  system  could  be  executed  in  parallel  with  the  three-dimensional  modelling 
system  as  it  provides  an  updated  two-dimensional  wire-frame  view  of  the  robot’s  orthogonal  environment. 
The  pattern  matching  process  w  ill  correlate  the  extracted  edges  from  the  image  with  the  known  features  from 
the  model. 

The  task  of  pose  determination  will  follow  the  matching  of  the  three  most  significant  edges  with  all 
possible  vertical  model  lines.  The  observed  difference  of  horizontal  viewing  angles  between  the  vertical 
image  edges  will  be  fitted  with  the  map  locations  of  vertical  model  lines  to  determine  the  only  possible 
position  and  orientation  of  the  robot’s  camera.  Many  different  possible  values  for  the  robot’s  pose  will  be 
calculated  as  different  line  matchings  are  tested. 


A  method  for  pose  verification  will  be  necessary  to  select  the  best  possible  pose  (hence,  also  the  best 
matching )  to  be  returned  as  the  correct  pose  of  the  robot.  The  correct  pose  can  then  be  used  to  update  the  pose 
maintained  by  dead-reckoning.  Sensor  fusion  with  the  present  sonar  system  could  possibly  follow  for 
confirming  the  accuracy  of  the  visual  fix  for  the  pose. 
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rv.  GENERATING  REGIONS  OF  EDGES 


A.  GRADIENT  IMAGE 

The  first  requirement  for  the  image  edge  extract  the  determination  of  high  contrast 

regions  between  the  areas  of  similar  light  intensity  whhu.  regions  of  significant  light 

intensity  contrast  will  define  the  edges.  The  CCD  camera  simphiie  ;on  of  light  intensity  by 

discretization  of  the  image  into  separate  pixels.  Chapter  VII  identifies  sou.  its  provided  by  the 

camera  and  used  in  die  graphics  workstation.  The  pixel  format  utilized  herein  rmat  which 

specifies  the  red,  green,  and  blue  color  attributes  of  the  pixel  and  is  described  fully  in  Chap  : Terences 

between  the  areas  of  the  image  are  therefore  determined  by  differences  in  pixel  intensities.  The  simplest  way 
to  consider  pixel  intensities  is  in  terms  of  grayscale  values  vice  considering  the  separate  associated  color 
attributes  of  each  pixel.  The  color  to  grayscale  conversion  of  RGB  pixels  in  also  described  in  chapter  Vll. 
Determining  the  differences  in  grayscale  intensities  between  adjacent  pixels  in  a  two-dimensional  image  will 
result  in  a  gradient  value  for  each  pixel.  The  pixel  gradient  is  a  vector  whose  magnitude  represents  the  amount 
of  light  intensity  change  between  adjacent  pixels,  and  whose  angular  orientation  is  directed  towards  the 
lighter  pixel.  Pixel  gradient  values  will  be  determined  in  a  two-dimensional  cartesian  coordinate  plane  by  the 
amount  of  change  of  grayscale  intensities  in  both  the  horizontal  (dx)  and  vertical  (dy)  directions  of  the  image. 

1.  Gradient  Window  Operators 

A  common  method  to  determine  pixel  gradients  is  by  the  use  of  gradient  window  operators.  Two 
gradient  window  operators,  one  for  determining  the  change  of  pixel  intensities  in  the  horizontal  (dx)  direction 
and  another  for  the  vertical  (dy)  direction,  must  be  specified.  Gradient  window  operators  arc  square  matrices 
of  weights,  mapped  onto  a  group  of  pixels  about  a  central  pixel  "-r  ghts  are  multiplied  with 

the  intensities  of  the  surrounding  pixels  and  then  ..c  intensity  changes  in  the 

horizontal  and  vertical  axes.  Commonlv  use  ^aare,  Prewitt,  Sobel,  and  Schirai 

(Figures  4. 1  through  4.4). 
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Figure  4.1.  The  Four-square  gradient  windows. 


Figure  4.2.  The  Prewitt  gradient  windows. 
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Figure  4  3.  The  Sobel  gradient  windows 
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Figure  4  4.  The  Schirai  gradient  windows. 

To  be  geometrically  correct  for  a  two-dimensional  plane  of  pixels  evenly  spaced  in  both  horizontal 
and  vertical  directions,  a  modified  Sobel  operator  should  be  used  with  values  of  Ji  vice  2  for  weights  of  the 
non-diagonal  pixels  (Figure  4.5).  This  would  assume  that  all  of  the  pixels  sensors  on  the  CCD  array  in  the 
camera  are  in  a  perfectly  square  grid.  However,  since  one  of  the  assumptions  of  this  vision  system  is  to  operate 
in  an  orthogonal  environment,  the  standard  Sobel  operator  could  prove  to  be  more  valuable  since  it  enhances 
the  vertical  and  horizontal  edges  found  in  the  orthogonal  environment.  After  implementing  and  testing  all  of 
these  window  operators,  the  Sobel  seemed  to  provide  the  strongest  edges  and  the  least  amount  of  smaller, 
insignificant  and  undesirable  edges . 
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Figure  4.5.  The  modified  Sobel  gradient  windows. 
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Once  the  values  for  dx  and  dy  are  calculated  by  the  gradient  window  operators,  the  gradient  is 
easily  calculated.  Pixel  gradient  magnitude  ( | V  (pixel t) | )  and  gradient  angle  ( V  a.(pixel;) )  would  then  be 
determined  by  equations  4. 1  and  4.2.  The  atan2(dy,dx)  function  is  common  in  many  programming  languages’ 
standard  math  libraries  and  is  described  in  Appendix  A. 


V  (pixel t)  |  = 

j(dX')2+  (dy,)2 

(4.1) 

Vaipixel;)  s 

atan2  ( dy{,  dxt) 

(4.2) 

To  determine  if  a  pixel  is  part  of  an  edge  between  two  areas  of  an  image,  the  gradient  magnitude 
must  be  greater  than  a  specified  threshold  value,  Cj.  Such  a  threshold  can  be  determined  dynamically  by 
scanning  through  the  image  once  to  determine  the  average  weight  of  pixel  intensities  or  by  maintaining  a 
histogram  of  previous  images'  average  pixel  intensities.  An  alternative  approach  is  to  keep  the  gradient 
magnitude  threshold  value  as  static  value,  determined  by  testing.  This  method  would  be  suitable  for  a  robot 
operating  in  an  environment  with  relatively  constant  illumination  and  alleviates  the  requirement  of  an  extra 
scan  through  the  image  to  dynamically  determine  a  value  for  Cj.  This  method  will  be  pursued  based  upon  the 
assumption  that  the  robot  will  initially  be  tested  in  an  indoor  environment  with  relatively  uniform  lighting, 
whereas  the  other  method  for  threshold  calculation  would  be  more  suitable  for  the  vision  system  adjusting  to 
major  environment  lighting  changes  (e  g.  the  robot  travelling  into  an  unlit  room). 


2.  Construction  of  the  Gradient  Image  with  the  Sobel  Operator 

The  Sobel  gradient  window  operators  will  be  utilized  to  calculate  pixel  gradients  from  the 
grayscale  values  for  each  pixel.  The  Sobel  operator  is  defined  as  a  two-dimensional  matrix;  that  is  to  be 
implemented  upon  pixels  read  and  stored  in  a  one-dimensional  array  (see  Chapter  VIII  for  description  of  the 
data  structure  for  the  image).  Using  the  image’s  horizontal  dimension  (x size)  as  an  offset  for  finding  the 
adjacent  pixels  in  the  neighboring  rows  of  a  two-dimensional  image,  equations  for  calculating  the  dx  and  dy 
values  for  the  gradient  of  a  central  pixel,  i.  are  defined  by  equations  4.3  and  4.4.  A  pointer  (ptr ).  is  utilized  to 
access  each  grayscale  pixel  value  from  the  one-dimensional  array  in  the  data  structure  for  the  image. 

-  ptr  [  i  +  xsize  -  1  ]  +  ptr  ( /  +  xsize  +  1  ] 

dx  =  -  2 ptr  [i  -  1  ]  +  2 ptr  [ i  +  1  ]  (4.3 ) 

—  ptr  [i  -  xsize  -  1]  +  ptr  [i  -  xsize  +  I] 
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(4.4) 


ptr  [i  +  xsize  -  1]  +  2ptr  [i  + xsize]  + ptr  [i  + xsize  +  1] 

dv  = 

-  ptr  [i- xsize  -  1]  -  2ptr[i  -  xsize]  -  ptr  [i  -  xsize  +  l] 

Notice  that  in  order  to  calculate  the  Sobel  operator  for  pixel  /',  pixel  i+xsiie+1  and  all  previous 
pixels  must  have  a  grayscale  values  already  determined.  A  possible  method  could  be  to  first  construct  a 
corresponding  grayscale  image  of  the  input  (color)  image  followed  by  construction  of  the  gradient  image 
using  the  Sobel  operator  equations.  Pixel  gradient  m  "cnitudes  larger  than  the  predefined  threshold  (C;)  would 
then  be  set  to  be  black  in  the  gradient  image  an  !  ”  ~  would  be  white.  This  would  provide  a  gradient 
image  such  as  displayed  in  Figure  3.4.  A  simple  alg  •  would  be: 

-  Read  in  input  image. 

-  Allocate  memory  for  new  grayscale  and  gradict< 

-  Buiki  the  grayscale  image  by  grayscale  conversion 

-  Build  the  gradient  image  via  Sobel  operator  for  all  pixels. 

-  If  pixel  gradient  magnitude  is  greater  than  the  magnitude  thresulu.  C,  .then  set  the 
corresponding  pixel  in  the  gradient  image  black,  else  set  the  corresponding  pixel  white. 

3.  When  Not  to  Use  the  Sobel  Operator 

However,  not  every  pixel  in  the  input  image  (more  precisely,  the  grayscale  image)  can  have  their 
associated  gradient  values  properly  calculated.  The  outermost  pixels,  those  in  the  top  and  bottom  rows  and 
the  leftmost  and  rightmost  columns  will  not  have  gradient  values  since  the  Sobel  operator  requires  eight 
adjacent  pixels  about  the  central  pixel  gradient  value.  Calcuf  afire  pradient  for  these  outermost  pixels  will  only 
produce  errors  when  using  the  gradients.  Therefore  ■  eut  image  are  two  pixels  less 

in  both  horizontal  and  vertical  directions  and  .  ,_.c  paid  towards  scanning  through 

the  input  image  in  order  not  to  calculate  pixel  gradients  for  uie  ,  utermost  pixels. 


B.  FINDING  CONTIGUOUS  REGIONS  OF  PIXELS  TO  DESCRIBE  EDGES 

Since  the  black  pixels  in  the  gradient  image  represent  pixels  with  gradient  magnitudes  sufficiently 
large  to  be  considered  part  of  an  edge,  each  black  pixel  must  be  considered  for  grouping  into  contiguous 
regions  that  define  the  edges.  Therefore,  grouping  pixels  into  regions  will  be  performed  upon  every  pixel  with 
a  gradient  magnitude  larger  than  the  threshold.  Cj.  Determining  a  suitable  value  for  Cj  is  critical  to  the 
success  of  this  algorithm.  A  threshold  that  is  too  high  will  not  allow  for  the  detection  of  all  significant  edges 
with  those  that  are  found  being  broken  and  incomplete.  On  the  other  hand,  a  threshold  that  is  too  low  lead  to 
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edge  finding  and  linear  fitting  of  non-edge  areas  of  the  image.  Chapter  VII  shows  results  of  varying  the 
gradient  magnitude  threshold  Cj. 


After  a  suitable  threshold  is  determined  and  used,  the  problem  of  grouping  edge-related  pixels  is  to 
define  how  two  pixels  share  features  common  to  an  edge.  Figure  4.6  can  be  considered  as  a  portion  of  a 
gradient  image  with  two  distinct  areas  of  common  light  intensity  (areas  A  and  B)  and  the  subsequent  edge  in 
between,  comprised  of  pixels  with  gradient  magnitudes  greater  than  the  given  threshold.  In  order  to  describe 
the  edge,  each  pixel  must  be  grouped  into  one  contiguous  region.  There  are  three  important  aspects  of  a  pixel 
that  can  be  used  for  this  comparison:  pixel  gradient  magnitude,  pixel  gradient  angle,  and  pixel  adjacency. 
Pixel  gradient  magnitude  has  already  been  considered  by  testing  with  threshold  Cj. 

line  segment  describing 


Figure  4.6.  A  region  of  pixels  defining  an  edge  in  the  gradient  image.  A  and  B  are  areas  of 
common  light  intensity  (pixels  that  are  white  in  the  gradient  image).  Area  B  is  darker  than 
A,  subsequently  all  edge  pixel  gradients  will  be  directed  towards  A.  Similarly  all  edge  pixel 
gradient  angles  will  be  close  to  the  normal  of  the  line  segment  describing  the  edge. 


For  two  pixels  to  be  of  the  same  edge,  they  must  both  have  gradient  angles  that  are  close.  "Closeness” 
can  be  defined  by  ensuring  the  difference  between  two  angles  is  less  than  some  constant  angle.  Cj.  The  pixel 
gradient  orientation  is  the  normal  to  the  tangent  of  the  border  between  the  light  and  dark  areas  of  the  image 
at  that  point  (areas  A  and  B  in  Figure  4.6  >.  Therefore,  all  pixel  gradient  orientations  will  be  close  to  the  normal 
of  the  line  segment  describing  the  edge  region.  If  a  pixel  's  gradient  angle  is  not  close,  then  the  pixel  can  not 
be  considered  to  be  part  of  that  edge.  The  effects  of  varying  C%  are  shown  in  Chapter  VI] . 


The  adjacency  requirement  can  readily  be  seen  in  Figure  4.6.  For  a  contiguous  region  of  pixels  to 
define  an  edge,  all  pixels  in  the  edge  must  be  adjacent  to  at  least  one  other  pixel  in  the  edge.  Thus,  after  testing 
a  pixel's  gradient  magnitude  for  description  of  the  gradient  image,  only  the  pixel’s  gradient  angle  and  the 
pixel’s  adjacency  with  a  previously  edge-defined  pixel  needs  to  be  considered  (Figure  4.7).  If  these 
requirements  fail,  the  pixel  must  still  be  considered  being  a  member  of  an  edge  and  therefore  it  will  be  the 
first  pixel  to  define  a  new  edge. 


row  presently  being  scanned 


Figure  4.7.  Pixel  /  is  only  compared  with  four  adjacent  pixels  to  check  gradient  orientation 
closeness.  One  of  these  four  pixels  must  already  belong  to  a  defined  edge. 


C.  DECIDING  WHEN  TO  STOP  ADDING  PIXELS  TO  AN  EDGE 

Continually  declaring  new  edges  for  every  pixel  that  does  not  meet  u.  bership  requirements  for 
a  declared  edge  will  rapidly  decrease  performance  as  the  number  of  edges  gets  large.  It  is  desirable  to  process 
edges  for  linear  fitting  once  it  has  been  determined  that  no  more  pixels  could  possibly  be  included.  This  is 
done  by  enforcing  the  eight-pixel  adjacency  rule  for  pixel  inclusion  in  an  edge.  Every  time  that  the  scan  of 
pixels  starts  at  a  new  row  in  the  image,  all  edges  must  be  inspected  to  ensure  that  the  last  pixel  added  to  each 
was  from  the  row  just  scanned  (Figure  4.8).  If  this  check  fails  tb-  found  on  the  present  row  (the 

row  to  be  scanned)  could  be  included  with  that  edge,  there:  .an  be  considered  completed  for 

grouping  of  contiguous  pixels  and  may  be  processed  for  linear  fitting.  Thus  all  edges  that  pass  this  test  are 
active  edges  that  can  still  accept  more  pixels  for  inclusion. 
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row  just  scanned 


Figure  4.8.  Prior  to  starting  the  scan  of  a  new  row,  the  list  of  edge  regions  must  be  reviewed  to  ensure 
that  the  last  pixel  added  to  each  edge  occurred  in  the  row  just  scanned.  If  not,  pixels  found  on  the  new 
row  could  not  possibly  be  associated  with  the  edge.  Thus,  edge  A  would  be  removed  from  the  edge 
list  whereas  edge  B  is  retained  as  an  “active"  edge  that  coula  still  accept  more  pixels  for  inclusion. 
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V.  LINE  FITTING  BY  LEAST  SQUARES 


A.  LEAST  SQUARES  FIT  METHOD 

One  method  for  determine  a  line  segment  that  adequately  describes  a  two-dimensional  region  of 
datapoints  is  by  least  squares.  Linear  fitting  by  least  squares  is  a  simple  and  efficient  method  for  calculating 
thousands  of  datapoints.  In  an  image,  the  least  squares  fit  moments,  described  by  Kanayama  [KAN 90b],  of 
pixel  locations  (x.y  pixel  coordinates  of  the  image)  for  all  pixels  in  the  edge  region  will  yield  an  associated 
line  segment  for  the  region’s  major  axis.  Verification  of  the  region  is  required  to  ensure  that  the  least  squares 
fit  moments  are  representative  of  a  line  segment.  Regions  that  meet  the  requirements  for  a  line  segment  are 
saved  for  the  desired  output  and  all  others  are  discarded. 

Once  the  decision  has  been  made  to  include  a  pixel  as  a  member  of  an  edge  region,  the  least  squares 
fit  moments  must  be  updated  for  the  region  to  include  the  new  pixel.  Consider  the  image  as  a  cartesian 
coordinate  system,  discretely  numbered  by  pixels  in  both  horizontal  and  ve-tical  directions.  Each  edge  region, 
R.  can  be  considered  as  a  collection  of  pixels  occupying  a  two-dimensional  space  (Figure  4.6).  Pixels’ 
positions  (numbered  by  pixels  in  x  and  y  axes)  are  used  to  define  the  least  square  fit  moments  mjtr 


1.  Primary  Least  Squares  Fit  Moments 

Each  moment  m]k  (0  Sj,  k  <  2  and  j  -f  A  <  2)  of  an  edge  region,  R,  is  defined  as: 


n 


i  =  1 


Considering  that  m00  is  equal  to  n,  m00  defines  the  number  of  pixels  associated  with  region.  R. 
The  centroid  C  of  an  edge  region,  R,  is  defined  as 


(5.1) 


(5.2) 


2.  Secondary  I  <-• 
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about  the  centroid  are: 
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M11S  X  =  mn~ 


miomoi 


3.  Parametric  Line  Fitting  of  the  Principle  Axis 


equation 


Using  a  parametric  representation  (r, a)  of  a  line  L,  a  point  at  coordinates  (x,y)  satisfies  the 


-K  n 

r  -  xcosa  +  ystna  (— <a<-).  (5.6) 

The  residual  of  pixel  i  (located  at  (.v,.  v())  and  the  line  L,  defined  by  8,,  is  the  distance  perpendicular 


to  L  and  pixel  i. 


8,  =  (Hr  -  .v()  cosa  +  (|ix  -  v()  sina. 


The  projection  of  pixel  i.  p' ..  onto  the  line  L  is 

p,'  =  ( A,  +  5f  cosa.  yr  +  5. sina) .  (5.8) 

Therefore,  p  ’  j  and  p '  n .  the  projections  of  the  first  and  last  pixels  associated  with  the  edge  region  R  onto  the 

line  segment  L,  can  be  used  as  estimates  of  the  endpoints  of  L. 

The  sum  of  all  the  residuals  is  then 
n 

S  =  (r  -  a,  cos  a  -v.cosa) 2.  (5.9) 


The  line  that  best  fits  the  set  of  points  will  minimize  the  sum  S.  Thus,  the  optimum  line  L  (/•,  a)  must  satisfy 

dS  dS  n 

T  =  -y-  =  0.  (5.10) 

dr  d  a 
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Therefore, 
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B.  TESTS  FOR  VALIDITY  OF  AN  EDGE  REPRESENTING  A  LINE  SEGMENT 

The  equivalent  ellipse  of  inertia  (Figure  5. 1 )  for  the  edge  region,  R,  will  have  the  same  moments  about 
the  centroid  (A/2o>  A/(  j,  and  A/02)  as  R-  A/ major  a™!  Mmin)n  the  moments  about  the  major  and  minor  axes  of 
the  ellipse,  are  defined  as: 


..  (^20  +  J^02)  (^02“^2(>)  .  .2 
M  major  =  - J - V - 4 -  +MH' 


(5.16) 


(A/20  +  ^(j2) 

A/. _  =  - - - — - (- 


( A/02  ^20) 


+  < 


2  '  \  4 

The  lengths  of  the  major  and  minor  axes,  dmajor  and  dmtnor,  are  defined  to  be: 


d  =4 

major 


\M 


major 
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(5.17) 


(5.18) 


d  =4 

minor 


i  ^ minor 


(5.19) 


Figure  5.1.  The  major  and  minor  axes  of  the  equivalent  ellipse  of  inertia  for  the 
region  of  pixels  defining  an  edge 
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For  this  implementation  of  least  squares  fit.  the  major  axis  will  be  considered  to  be  the  associated  line 
segment  for  die  edge  region.  The  endpoints  of  the  major  axis  will  be  the  endpoints  for  the  desired  line 
segment.  The  projections  of  the  first  and  last  pixels,  p'  j  and  p'  n ,  associated  with  the  region  R  will  therefore 


be  used  as  estimates  for  the  two  endpoints  of  L.  A  ratio,  p,  of  the  axes  length  can  then  be  used  to  describe  the 
thinness  of  the  ellipse. 


P  = 


major 


(5.20) 


Using  the  values  for  number  of  pixels  (mw),  major  axis  length  and  ellipse  thinness  (p)  as 

parameters  for  comparison  to  edge  significance,  edge  length  and  how  much  the  edge  resembles  a  line  can  be 
simply  tested  as  long  as  the  least  squares  fits  moments  are  maintained  for  every  pixel  included  in  an  edge.  Let 

Cj  (0  £  C3  <  1)  bea  constant  for  the  maximum  allowable  ratio  p  that  car.  be  used  to  described  a  line.  Let 
C4  be  a  specified  constant  for  the  minimum  number  of  pixels  and  C5  for  the  minimum  line  length.  Three 
requirements  can  therefore  be  specified  for  the  line  testing  -  ?  m  edge  region. 

1.  the  ratio  of  axes  lengths  less  than  the  maxu.  ratio  (p  <  Cj), 

2.  the  number  of  pixels  greater  than  the  specified  minimum  ( moo  >  C4),  and 

3.  the  line  length  of  the  region  greater  than  the  minimum  length  (dma^or  >  C5). 

The  ratio  p  proves  to  be  the  most  significant  measurement.  For  p  to  equal  1.0  means  that  the  length  of 
the  minor  axis  is  equal  to  the  length  of  the  major  axis,  representing  an  edge  region  that  resembles  a  circular 
blob.  Therefore,  p  can  be  compared  to  a  maximum  ratio,  Cj,  specified  by  the  user.  Cj  equal  to  1.0  allows  all 
edge  regions  to  be  considered  thin  enough  to  represent  a  line  segment.  A  value  of  0.1  seems  to  work  well. 

The  two  other  tests  can  be  used  to  trim  down  the  number  of  smaller,  less  significant  edges  found. 
Specifying  minimum  number  of  pixels  in  an  edge  and  minimum  edge  length,  C4  and  C5  respectively,  is  an 
effective  means  to  filter  out  less  significant  line  segments.  The  region'  meet  these  requirements  will  be 
saved  for  the  desired  output  as  the  found  line  segments  and  all  r!'  ns  will  be  discarded.  The  effects  of 

varying  the  parameters  C3 ,  C4,  and  C  t  are  described  in  Ch 
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VI.  ALGORITHM  FOR  EDGE  EXTRACTION 


From  the  principles  stated  in  the  previous  chapters,  the  following  elements  were  identified  to  be 
incorporated  the  edge  extraction  algorithm: 

1.  Determine  which  pixels  compose  the  edges. 

2.  Group  pixels  together  into  regions  describing  individual  edges. 

3.  Describe  a  completed  edge  region  in  terms  of  a  line  segment  via  least  squares  fit  of  pixels' 
positions. 

4.  Ensure  that  line  segment  representation  of  the  edge  meets  the  specified  thinness  and  length 
requirements. 

The  implementations  are  coded  in  the  ANSI  version  of  the  C  language  and  are  contained  in  Appendix  B.  A 
user’s  guide  for  compiling  and  executing  the  implementations  is  in  Appendix  D. 

A.  THE  FINDEDGE  METHOD 

Findedfie  is  a  C-program  that  follows  the  above  outline.  Remembering  that  the  grayscale  value  for 
pixel  i+.xsize+l  must  be  determined  prior  to  the  gradient  calculation  for  pixel  r,  the  algorithm  fo rfindedge 
can  be  stated  to  build  both  the  grayscale  and  gradient  images  in  one  pass  of  scanning  the  input  (color)  image. 
This  algorithm  will  ensure  that  gradient  values  are  not  calculated  for  the  outermost  pixels.  Pixels  with 
gradient  magnitudes  greater  than  the  specified  threshold,  C,,  will  be  considered  as  datapoints  for  the  edges. 

Two  tests  were  identified  for  a  an  edge-related  pixel  to  be  included  as  a  member  of  a  particular  edge. 
Those  tests  were: 

1  The  pixel  must  be  adjacent  to  a  pixel  already  included  in  an  edge. 

2.  The  pixel's  gradient  angle  must  be  ‘  close”  to  that  of  the  averaged  angle  of  adjacent  pixel’s. 

Two  angles  are  "close"  if  the  difference  between  them  is  less  than  a  maximum  constant.  C:.  These  tests 
can  be  conducted  immediately  once  a  pixel  has  been  identified  to  be  an  edge-related  pixel.  Therefore,  all 
black  pixels  in  the  gradient  image  will  be  tested  for  edge  membership. 

Pixels  meeting  the  edge  membership  requirements  must  then  be  incorporated  with  the  edge  by 
updating  the  least  squares  fit  moments.  mjh  for  the  edge  region,  R.  This  is  accomplished  by  determining  the 
rth  pixel  s  two-dimensional  image  position  (x.y)  and  summing  the  moments.  mfh  with  the  corresponding 
weights.  If  a  p;xel  fails  the  membership  test  for  the  declared  edges,  then  that  pixel  will  be  used  as  the  first  to 
declare  a  new  edge  Thus,  a  linked  list  is  the  preferable  data  structure  to  maintain  the  declared  edge  regions 
Observing  that  new  edges  are  declared  as  a  linear  scan  of  the  image  is  conducted,  and  that  the  first 
requirement  for  pixel  membership  with  an  edge  is  adjacency,  a  pixel  needs  only  to  conduct  the  membership 
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test  with  the  pixel  to  the  immediate  left  (assuming  a  left-to-right,  bottom-to-top  scan).  This  builds  edge 
regions  for  the  present  row  only.  Prior  to  continuing  the  scan  at  the  start  of  the  next  row,  the  list  of  edge 
regions  for  the  row  just  scanned  will  be  compared  with  the  list  of  edge  regions  from  the  previous  row  to  see 
if  they  can  be  combined  based  upon  vertical  adjacency  of  edge  regions  between  the  two  rows  and  “closeness" 
of  average  orientation  of  pixels’  gradient  angles. 

Maintaining  values  for  the  first  and  last  pixels  included  for  each  edge  is  necessary  for  the  determination 
of  line  segment  endpoints.  Keeping  track  of  first  and  last  pixels  added  is  straightforward  with  only  one 
exception  for  horizontally  oriented  lines.  The  case  for  horizontal  lines  (Figure  6.1)  requires  determination  of 
endpoints  based  upon  the  leftmost  and  rightmost  pixels  of  the  edge  vice  the  first  and  last  pixels  added. 


General  Case  of  Determining  Line  Segment  Endpoints 

last  pixel 


CD 
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E 

o 
c n 

a 


e 

2 


line  segment 

^prVeaion  of  pixels,  R,  defining  an  edge 

first  pixef" 


horizontal  axis  of  image  (x) 

Horizontal  Case  for  Line  Segment  Endpoints 


®  leftmost  line  segment 

?  >»<  /-  ...... 


last  pixel 


rightmost  pixel 


region  of  pixels,  R,  defining  an  edge 


horizontal  axis  of  image  (x) 


Figure  6. 1 .  General  and  horizontal  cases  for  determining  pixels  to  be  used  for  calculation  of 
line  segment  endpoints.  The  first  and  last  pixels  in  the  horizontal  case  may  not  necessarily 
fully  describe  the  extent  of  the  associated  line  segment,  in  which  case  the  leftmost  and  right¬ 
most  pixels  should  be  used. 
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Edges  from  the  previous  row  that  are  not  appended  with  pixels  from  the  row  just  scanned  are 
considered  finished  with  their  construction  and  can  therefore  be  tested  for  meeting  the  three  requirements  for 
a  line  segment.  These  requirements  were  identified  to  be: 

1 .  The  ratio  of  lengths  of  major  and  minor  axes  (p)  <  C,. 

2.  Number  of  pixels  in  an  edge  (moo)  >  CV 

3.  The  length  of  the  major  axis  (dma]0r)  >  C,. 

Determining  dmajor  and  p  is  easily  done  since  the  primary  least  square  fit  moments  are  updated  for 
every’  pixel  added.  Edges  that  meet  these  requirements  will  serve  as  the  desired  output  of  straight  edges, 
otherwise  they  are  discarded  and  the  associated  memory  is  freed. 

B.  AN  ATTEMPT  TO  CUT  COMPUTATION  COSTS 

The  findedge  algorithm  requires  an  averaged  value  of  the  pixel  gradient  angle  orientations  to  be 
maintained  for  each  edge.  This  average  value  is  used  for  the  comparison  with  pixel  gradient  orientation  to 
determine  angle  “closeness.”  Thus,  for  every  pixel  added  to  an  edge,  the  average  of  the  pixels'  gradient 
orientation  must  be  calculated.  To  alleviate  these  computations,  the  specified  maximum  value  for  angle 
closeness.  C2,  could  be  used  to  determine  two  static  sets  of  angles  that  any  pixel  gradient  orientation  could  be 
defined  by  These  two  sets  could  be  thought  of  as  dividing  up  all  possible  angles  between  n  and  -rr  by  C2  as 
illustrated  in  Figure  6.2. 

"Closeness”  could  now  be  defined  by  comparing  the  two  groups  that  a  pixel  gradient  angle  belongs  to 
with  the  groups  of  the  adjacent  edge-related  pixel’s  gradient  angle.  If  either  groups  are  similar,  then  the 
gradient  angles  are  close.  This  method  is  termed  static  since  the  two  sets  of  angles  are  computed  only  once 
and  remain  unchanged  whereas  the  findedge  method  continually  recomputed  the  average  gradient  angle  of 
the  pixels  in  the  edge  region.  The  results  of  this  method  of  pixel  grouping  into  straight  line  edges  are  shown 
in  Chapter  VII.  Compared  with  the  line  segments  from  the  findedge  algorithm,  more  line  segments  are 
extracted  but  shorter  and  more  fragmented.  Since  the  desired  output  is  a  simple  structure  (line  segments)  that 
best  represents  the  linear  features  in  an  image,  the  performance  of  the  static  method  is  worse. 
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Division  of  angles  by  C2  -  20.0  degrees 


Angie  Group  1  Angle  Group  2 


Figure  6.2.  The  >c  groups  of  angles  detemined  for  C3  -  2(.'.l  k-snees.  Each  pixel 

gradient  orientati  n  i .  Mong  to  one  region  from  each  group  of  angles.  7  l>c  groups  can 
be  thought  of  as  overlap  ng  in  such  a  way  so  that  “close”  gradient  angles  not  belonging 
to  the  same  region  in  one  group  will  belong  to  the  same  region  in  the  other  group. 


C.  IGNORING  GRAYSCALE  CONVERSION  WITH  THE  FASTEDGE  METHOD 

Computation  costs  can  still  be  reduced  in  the  frndtdgt  algorithm  by  omitting  the  construction  of  the 
grayscale  and  gradient  images  Observing  that  the  green  component  of  the  RGB  color  format  accounts  for  the 
majority  of  grayscale  weights,  pixel  gradients  can  be  estimated  by  using  only  the  green  component  value.  This 
is  accomplished  with  a  bitmask  to  get  only  the  green  color  values  for  use  with  the  Sooel  operator  (equations 
4.3  and  4.4).  Thus,  construction  of  the  grayscale  image  is  unnecessary  Si  rurally,  by  observing  that  the 
gradient  image  only  identifies  those  pixels  that  descriS-  .  ..  ise  not  required  for  the  construction 

of  the  edges,  its  construction  and  storage  as  a  real  image  will  be  omitted. 

Therefore,  the  program  fastedge  can  perform  the  same  straight  line  edge  extraction  as  findedge, 
however,  since  the  values  for  pixel  gradient  calculation  are  only  the  green  color  components  and  not  the 
weighted  grayscale  sum,  a  new  gradient  magnitude  threshold  value  (C,)  must  be  determined.  The  fastedge 
implementation  displays  the  input  (color)  image  and  draws  the  extracted  lines  over  the  image.  Results  are  in 
Chapter  VII.  A  guide  for  compiling  and  using  fastedge  is  contained  in  Appendix  D 
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VII.  IMPLEMENTATION 


A.  SYSTEM  DESCRIPTION 

The  graphics  workstation  used  for  feature  extraction  is  a  SiliconGraphics  Personal  Iris.  Its  graphics 
support  libraries  are  implemented  in  the  C  language;  therefore,  the  code  for  the  findedge  and  fastedge  methods 
is  in  C.  The  VideoFramer  frame  grabbing  hardware  is  a  daughterboard  on  the  Iris  that  accepts  the  pixel 
information  after  it  has  been  converted  from  the  video  source  signal  to  an  acceptable  pixel  format  (e.g.  RGB). 
A  typical  image  from  the  charged-couple  device  (CCD)  television  camera  is  646  x  486  pixels.  That  is  3 13,956 
total  pixels  to  be  stored  in  a  file  approximately  1.6  Mbytes.  Efficient  algorithm  design  is  paramount  for  such 
large  image  files.  The  Personal  Iris  is  a  35  MHz  machine  that  can  read  an  image  file  and  conduct  edge 
extraction  via  the  fastedge  implementation  in  approximately  15  seconds  (about  3  seconds  for  an  image 
“shrunken"  to  half  of  its  dimensions). 

B.  THE  INPUT  IMAGE 

Prior  to  encoding  an  algorithm  to  find  the  straight  line  edges  in  an  image,  a  careful  analysis  of  the 
image  must  be  considered.  The  CCD  camera  used  (JVC  model  TK870U)  can  provide  a  video  signal  in  various 
formats.  Some  of  these  video  formats  include: 

Betamax  Video 

-  Dl  Video 

-  NTSC  (National  Television  Systems  Committee  format) 

-PAL 

-  RGB  (red.  green,  blue  attributes) 

-  Super  VHS  Video 

There  are  also  various  formats  for  saving  and  displaying  images  as  files  on  a  computer.  Some  of  these 
formats  include: 

-  Apple  Macintosh  MacPaint  file 

-  CompuServe  Graphics  Interchange  Format  file  (GIF) 

-  Encapsulated  Postscript  file  <  EPS ) 

-  Hierarchical  Data  file 

-  PIXAR  picture  file 

-  Portable  Bitmap  file 

-  Postscript  file 

-  SiliconGraphics  Image  RGB  file  (SGI) 

-  Stardent  AVS  X  image  file 

-  Sun  Rasterfile 

-  Tagged  image  file  (Tib) 

-  Utah  Run-Length-Encoded  image  file 

-  Wavefront  raster  image  file 

-  XI I  bitmap  file 
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The  RGB  formal  was  chosen  to  be  utilized  because  it  is  easily  translated  into  NTSC  (National 
Television  Standard)  and  exists  as  a  standard  format  for  both  the  CCD  camera  output  and  the  SiliconGraphics 
workstation. 

1.  The  RGB  Format 

The  RGB  format  is  a  32  bit  digital  format  to  describe  a  single  pixel  in  terms  of  its  basic  color 
components  (red,  green,  and  blue).  In  the  C  language,  a  32  bit  format  is  referred  to  as  a  “long  integer”  data 
type.  Eight  bits  arc  used  to  represent  each  of  the  intensities  for  the  red,  green,  and  blue  color  components  of 
the  pixel.  This  allows  for  intensity  values  to  range  from  0  to  2S5  decimal  (0  to  FF  hexadecimal)  for  each 
component  An  intensity  value  of  0  represents  zero  light  intensity  for  that  color  component,  25S  for  full 
illumination. 

The  RGB  format  uses  the  red,  green,  and  blue  colors  as  the  basic  components  because  they  are  the 
three  primary  additive  colors.  An  additive  color  scheme  used  on  television  and  similar  displays  combines 
colors  from  red,  green,  and  blue  light  sources  together  to  create  the  nest  of  the  colors  in  the  visual  spectrum. 
This  is  contrasts  with  the  subtractive  color  scheme  implemented  with  yellow,  cyan,  and  magenta  as  the  three 
primary  colors  which  is  useo  in  film  photography  for  filtering  a  white  light  source  to  create  the  other  colors. 

The  RGB  A  in  mat  only  differs  from  the  RGB  formal  in  that  it  has  eight  bits  reserved  for  the  alpha 
component  which  is  used  for  transparency  on  graphics  workstations.  An  alpha  component  with  a  value  of 
zero  is  completely  transparent,  whereas  a  value  of  255  is  completely  opaque.  The  RGB  and  RGBA  formats 
are  both  32  bits  long  and  are  basically  identical.  The  RGB  format  simply  ignores  the  alpha  component  and 
operates  as  if  all  colors  are  opaque  (alpha  value  set  to  FF  hexadecimal).  Figure  7.1  shows  the  bit  placement 
for  the  RGB/RGBA  formats.  In  order  to  allow  easier  translation  between  these  two  formats,  the  alpha 
component  is  set  to  FF  hexadecimal.  To  illustrate  this  format  with  some  examples,  the  color  black  is 
represented  as  FF000000  hexadecimal  (alpha  =  FF;  blue,  green,  red  =  00),  white  is  FFFFFFFF,  and  bright 
kelly  green  is  FF00FF00. 
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Figure  7.1.  The  RGB  /  RGBA  formats.  RGB  formal  ignores  the  alpha  block. 
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2.  Color  to  Grayscale  Conversion 


The  findedge  implementation  of  feature  extraction  requires  a  grayscale  image.  However,  if  a  color 

camera  is  used,  it  will  be  necessary  to  convert  the  color  image  to  a  grayscale  image.  The  RGB  format 

describes  each  pixel  in  terms  of  its  basic  additive  primary  colors,  whereas  the  overall  intensity  of  the  attributes 

would  be  a  grayscale  value.  To  determine  the  grayscale  value,  each  of  the  color  values  must  be  weighted  and 

summed.  The  weights  assigned  to  each  color  for  representation  as  a  black  and  white  television  (grayscale) 

pixel  have  been  standardized  by  the  National  Television  Systems  Committee.  The  following  weights  are 

prescribed  by  the  NTSC  [HAL89]. 

Red  =  0.299 
Green  =  0.587 
Blue  =  0.1 14 


Converting  a  color  pixel  to  a  grayscale  value  is  simply  assigning  the  sum  of  the  weighted  color 
component  values  to  the  grayscale  value  (equation  7.1). 


^  grayscale 
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(Eq  7.1) 


4.  The  NPSIMAGE  Data  Structure 

The  RGB  format  is  a  well  known  and  widely  used  format  for  defining  a  single  color  pixel. 
However,  formats  for  describing  entire  images  via  RGB  pixels  can  still  vary.  One  such  format,  a  simplified 
model  of  a  standard  video  image,  developed  by  Mike  Zyda  at  the  Naval  Postgraduate  School,  is  the 
NPSIMAGE  data  structure  outlined  in  the  image  types. h  file  in  Appendix  A.  NPSIMAGE  is  a  C  language 
data  structure  (a  record  type)  to  store  an  RGB,  RGBA,  or  color-mapped  image. 

For  all  video  images,  the  pixels  are  represented  as  a  single,  large,  one-dimensional  array  of  values 
starting  with  the  first  pixel  in  one  comer  of  the  two-dimensional  image  and  ending  at  the  opposite  comer 
(Figure  7.2).  All  pixels  in  this  array  are  arranged  via  a  single,  linear  scan  of  the  video  image  (i.e.  left  to  right, 
bottom  to  top).  In  order  to  properly  display  the  image  from  a  saved  format,  it  is  crucial  to  have  the  dimensions 
of  the  original  image  saved.  The  dimensions  of  the  input  image  are  expressed  in  number  of  pixels  in  the 
horizontal  and  vertical  directions.  NPSIMAGE  has  two  integer  value  slots,  xsize  and  ysize,  to  store  these 
dimensions. 


The  image  format  must  subsequently  provide  an  efficient  way  to  access  the  one-dimensional  array 
of  pixel  values.  This  is  generally  done  in  the  C  language  via  a  pointer  initialized  to  the  first  pixel  of  the  one¬ 
dimensional  array.  Moving  the  pointer  by  altering  its  offset  allows  easy  access  to  all  pixel  values.  When  using 
the  NPSIMAGE  data  structure,  such  a  pointer  will  be  initialized  to  the  imgdata  .bitsptr  long  integer 
array  (Figure  7.3). 


Figure  7.2.  Construction  of  one-dimensional  array  of  pixel  values  from  two-dimensional  video  image. 
Order  of  pixel  scan  is  left  to  right,  bottom  to  top,  as  used  in  the  NPSIMAGE  data  structure. 


NPSIMAGE  *img;  /*  a  pointer  to  the  image  data  structure  */ 

long  *ptr;  /*  a  pointer  to  pixel  values 

(represented  as  long  integers)  */ 

ptr  =  img->imgdata .bitsptr ; 

Figure  7.3.  Declaration  and  initialization  of  a  C  ...  .  pointr  he  RGB  values  in  an  NPSIMAGE. 
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VIII.  EDGE  EXTRACTION  RESULTS 


A.  FINDEDGE  IMPLEMENTATION 

The  implementation  of  ih c findcdge  algorithm  described  in  Chapter  VI  and  detailed  in  Appendix  B 
provides  extracted  line-segment  features  such  as  in  Figure  8.1.  The  line  segments  found  are  from  the  input 
image  of  the  chair  (.Figure  3.3).  The  five  parameters  for  gradient  thresholds  and  line  segment  properties  (C, 
through  C5)  were  determined  b>  trial  and  error.  The  effects  of  modifying  the  parameters  is  seen  in  later  in  this 
Chapter. 


Figure  8.1.  Lines  found  from  image  of  chair  vufindedge  implementation. 
CL  =  ,0(  >i).ii( >().(';  C-  =  20.0  degrees:  C3  =  0.1:  C4  =  20  pixels:  C5  =  20.0. 


B.  STATIC  METHOD  OF  GRADIENT  ANGLE  TESTING 

Results  from  the  static  method  for  determination  of  “closeness”  of  pixel  gradient  angle  orientation  are 
displayed  in  Figures  8.2  through  8.4.  The  input  image  is  that  of  the  chair  (Figure  3.3)  and  the  resultant  line 
segments  can  be  compared  with  Figure  8.1.  All  five  parameters  are  the  same  as  those  used  in  Figure  8.1. 
Figure  8.2  shows  the  lines  found  using  the  first  group  of  static  angles  divided  by  20  degrees  (see  Figure  6.2) 
and  Figure  8.3  shows  the  lines  from  the  second  group  of  static  angles.  Figure  8.4  displays  all  lines  found  from 
both  Figures  8.2  and  8.3.  Comparing  Figures  8. 1  and  8.4  demonstrates  the  shortcomings  of  the  static  method. 
The  line  segments  are  shorter,  broken,  and  subsequently  more  numerous,  all  of  which  are  traits  not  desired 
for  the  purposes  of  pattern-matching. 


Figure  8.2.  Lines  found  from  image  of  chair  via  static  gradient  angles  method  (group  1  of  Figure 
6.2).  C,  =  5,000,000.0;  C2  =  20.0  degrees;  C3  =  0.1;  C4  =  20  pixels;  C5  =  20.0. 
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Figure  8.3  Lines  found  from  image  of  chair  via  static  gradient  angles  method  (group2  of  Figure 
6.2).  Cl  *  5.000.000.0:  C2  =  20.0  degrees:  C3  *  0.1:  C4  -  20  pixels:  C5  -20.0. 


Figure  8.4,  All  lines  found  from  image  of  chair  via  static  gradient  angles  method 
C,  =  5.000.000.0;  C2  =  20.0  degrees:  C,  =  0.1:  C4  =  20  pixels.  C,  =  20.0. 


C.  FASTEDGE  IMPLEMENTATION 

The  last  algorithm  described  in  Chapter  VI,  the  fastedge  method  produced  lines  very  similar  to  those 
found  by  the  findedge  method  Figure  8.5  shows  the  resultant  line  segments  via  the  fastedge  implementation. 
The  major  difference  between  this  and  findedge  was  determination  of  the  gradient  magnitude  threshold  value 
C|.  Findedge  used  color  to  grayscale  conversion  for  an  absolute  intensity  value  whereas  fastedge  only  used 
the  green  color  component  of  the  RGB  image. 


Figure  8.5.  Lines  found  from  image  of  chair  via  fastedge  implementation. 
C|  =  30,000.0;  C2  =  20.0  degrees;  C3  =  0.1;  C4  =  20  pixels;  C5  =  20.0. 
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D.  EFFECTS  OF  EDGE  EXTRACTION  AFTER  “SHRINKING”  AN  IMAGE 


Condensing  an  input  image  to  half  of  its  dimensions  prior  to  performing  the  straight  edge  feature 
extraction  routines  produces  some  noteworthy  results.  Aside  from  requiring  only  a  quarter  of  the 
computational  time  for  edge  extraction,  the  resultant  line  segments  are  generally  fewer,  longer,  and  seem  to 
model  the  image  in  simpler  terms.  To  illustrate  such  differences,  Figure  8.6  is  resultant  line  segments  (via 
fasiedge )  of  the  chair  image  condensed  to  half  of  its  length  and  width  (quarter  of  the  original  area).  Its  results 
can  be  compared  with  the  lines  extracted  in  Figure  8.5.  Similarly,  Figure  8.7  is  a  view  of  a  hallway  interior 
with  its  extracted  line  segments  (Figures  8.8  and  8.9)  to  be  compared  with  the  edges  extracted  from  the  same 
image  after  shrinking  (Figure  8.10)  and  edge  extraction  (Figures  8.11  and  8.12). 


Figure  8.6.  Lines  found  from  image  of  chair  after  it  has  been  shrunken  to  half  of  its 
dimensions  and  using  the  fasiedge  program.  C|  =  30,000.0;  C2  =  20.0  degrees:  C3  =  0.1; 
C4  =  20  pixels;  C5  =  20.0. 
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Figure  8.7.  Image  of  hallway  interior. 


Cj  =  30.000.0;  C2  =  20.0  degrees;  C-*  =  0.1:  C4  =  20  pixels;  C*  =  20.0 
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Figure  8.9.  Extracted  edges  superimposed  over  image  of  hallway  interior. 
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Figure  8.10.  Shrunken  image  of  hallway  interior. 


Figure  8.1 1.  Extracted  edges  of  shrunken  K 

C?!  =  30,000.0;  C2  =  20.0  degrees;  C«  -  _,  =  20.U. 


Figure  8.12.  Extracted  edges  superimposed  over  image  of  hallway  interior. 


by  fastedge  method 
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E.  EFFECi  o  v  .  VARYING  FEATURE  EXTRACTION  PARAMETERS 


1.  Gradient  Magnitude  Threshold  Ct 

The  threshold  value  C)  is  used  to  determine  if  the  gradient  magnitude  of  a  pixel  is  sufficiently 
large  enough  to  describe  an  edge.  If  the  value  for  pixel  gradient  magnitude,  determined  by  equation  4.1,  is 
larger  than  Ct,  then  the  pixel  in  the  gradient  image  is  set  black,  otherwise  it  is  set  white.  In  the  findedge 
program,  the  value  for  C(  was  determined  to  be  5,000,000.0  by  testing.  Figures  8.9  through  8.1 1  show  the 
associated  gradient  images  for  values  of  Ct  equal  to  5M,  8M,  and  2M,  respectively. 
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Figure  8.10.  Gradient  image  of  a  chair  with  magnitude  threshold  C|  =  8,000,000.0. 


Figure  8.1 1.  Gradient  image  of  a  chair  with  magnitude  threshold  C(  =  2,000,000.0. 
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2.  Gradient  Angle  “Closeness"  Angle  Cj 


The  threshold  value  C„  the  maximum  difference  between  two  pixel  gradient  orientations  was 
tested  by  trial  and  error  to  produce  more  extracted  line  segments  that  best  represented  the  straight  edges  from 
the  input  image.  For  the  image  of  the  chair  (Figure  3.3),  C,  =  17.2  degrees  performed  best.  Larger  angles 
resulted  in  combining  multiple  straight  edges  from  the  image  into  one  extracted  line  segment,  whereas  smaller 
angles  were  too  restrictive  to  produce  long  line  segments.  The  best  Q  value  for  one  image  is  not  necessarily 
the  best  value  for  all  images.  Pictures  of  the  hallway  interior,  used  for  pattern-matching  and  pose- 
determination,  utilize  C2  -  28.65  degrees  (0.5  radians).  The  images  in  Figures  8.12  through  8.14  illustrate  the 
effects  of  modifying  C2  of  the  chair  image  via  the  fastedge  implementation. 


Figure  8.12.  Lines  extracted  from  image  of  chair  for  C2  =  17.2  degrees  (0.3  radians). 
C,  =  30,000;  C,  =  0. 1;  C4  =  20  pixels;  C5  =  20.0. 


43 


Figure  8. 13.  Lines  extracted  front  image  of  chair  for  C2  =  30  degrees. 
C,  =  30,000:  C,  =  0.  J :  C4  =  20  pixels:  C,  =  20.0. 


C,  =  30.000:  C,  =  0.1:  C4  =  20  pixels:  C5  =  20.0. 
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3.  Line  Test  Parameters  Cj,  C4,  and  C$ 


The  three  line  test  parameters  identified  in  Chapter  V  are  utilized  for  limiting  the  number  of 
extracted  line  segments  by  placing  restrictions  on  the  ratio  of  axes  lengths  (p  =  dn^d^j^),  the  number  of 
pixels  (moo),  and  the  major  axis  length  (d^joi),  respectively.  The  lines  extracted  from  the  chair  image  by 
modifying  C3,  C4,  and  C5  are  shown  in  Figures  8.15  through  8.17.  Each  Figure  can  be  compared  with  the 
baseline  extracted  features  in  Figure  8.12. 


Figure  8. 15.  Lines  extracted  from  image  of  chair  for  maximum  thinness  ratio  C3  =  0.05. 
C[  =  30,000;  C3  =  17.2  degrees;  C4  =  20  pixels;  C5  =  20.0. 
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Figure  8.16.  Lines  extracted  from  image  of  chair  for  minimum  number  of  pixels  C4  =  40  pixels. 
C,  =  30,000;  C2  =  17.2  degrees;  C3  =  0.1;  C5  =  20.0. 


Figure  8.17.  Lines  extracted  from  image  of  chair  for  minimum  axis  length  C5  =  50.0. 
Cj  =  30,000;  C2  =  17.2  degrees;  C3  =  0.1;  C4  =  20  pixels. 
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IX.  MATCHING  LINE  SEGMENTS 


The  focus  of  this  Chapter  is  the  development  of  a  matching  algorithm  that  compares  the  straight  edges 
extracted  from  the  input  image  with  the  linear  features  of  the  environment  model.  The  pattern  matching  of 
two-dimensional  line  segments  can  be  divided  into  two  different  problems: 

1.  matching  with  the  model  based  upon  a  known  location  and  orientation  of  the  robot  (the  pose 
or  configuration  of  the  robot), 

2.  and  matching  with  the  model  without  knowledge  of  the  robot’s  pose. 

The  first  problem  is  applied  to  position  correction  and  vehicular  navigation,  whereas  the  second 
problem  is  that  of  object  identification  and  viewing  aspect  determination.  The  first  problem  is  much  more 
constrained  than  the  second  and  is  pursued  herein  towards  the  goal  of  pose  determination  and  vehicle 
navigation.  Expecting  small  dead-reckoning  errors  to  arise  in  the  vehicle’s  pose,  the  matching  algorithm  must 
exhibit  some  degree  of  robustness  if  it  is  to  make  valid  corrections.  Therefore,  the  goal  of  the  matching 
process  is  to  find  the  best  match  of  image  lines  to  model  lines. 

A.  THE  ENVIRONMENT  MODEL 

The  testing  and  operating  environment  for  Yamabico-11  is  the  fifth  floor  of  Spanagel  Hall  at  the  Naval 
Postgraduate  School.  Modeling  this  environment  in  a  graphical  database  [STE92]  was  accomplished  in 
conjunction  with  the  above  edge  extraction  methods  for  the  pattern  matching  task  described  in  Chapter  IX. 
The  model  is  a  “2-D  +”  model  similar  to  that  described  by  David  Man  [MAR79].  The  environment  model  is 
designed  to  support  requirements  for  visual  navigation,  sonar  navigation,  shortest  distance  path 
determination,  and  safe  path  planning.  The  flooiplan  features  are  expressed  in  the  xy  cartesian  plane  and 
associated  heights  of  hallway  features  are  modelled  in  the  z  axis.  The  notion  of free  space  is  aided  by  defining 
polygons  in  the  xy  plane  as  either  floor  or  ceiling  polygons.  All  database  measurements  are  in  inches.  All 
support  routines  for  implementation  of  the  model  are  coded  in  ANSI  C  and  described  fully  in  [STE92], 

1.  Interfacing  the  Model  Database 

Since  the  application  features  of  the  environment  were  encoded  in  the  same  language  as  the  image 
processing  and  pattern  matching  methods,  accessing  the  model  is  quite  simple.  The  pattern  matching  routine 
will  invoke  calls  to  both  the  image  processing  and  environment  model  functions  to  access  the  line  segment 
primitives  utilized  in  the  matching  process.  The  primary  call  to  the  model  (get_view( ))  requires  providing  an 
estimated  pose  (position  and  orientation  information)  of  the  robotic  vehicle  and  the  focal  length  of  the  camera 
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lens.  The  function  returns  a  data  structure  that  includes  pointers  to  linked  lists  of  the  line  segments  of  the 
model  database  mapped  to  a  two-dimensional  viewing  plane  that  the  robot  should  expect  to  see  given  the 
input  pose  and  visibility  constraints  of  occluding  edges.  The  line  segments  shown  in  the  model’s  two- 
dimensional  view  represent  the  orthogonal  features  of  the  operating  environment,  primarily  the  junctures  of 
the  walls,  ceiling,  floor,  doors,  and  overhead  lights. 

B.  BASIC  COMPARISON  OF  LINE  SEGMENTS 

Since  the  environment  model  provides  a  two-dimensional  view  of  the  known  linear  features  for  a  given 
pose,  the  matching  algorithm  needs  only  to  determine  linear  matches  in  two  dimensions.  From  the  results  seen 
in  Chapter  VII,  the  extracted  edges  from  the  input  image  do  not  cross,  overlap,  or  touch.  The  extracted  linear 
edges  are,  at  best,  incomplete  segments  of  the  modelled  linear  features.  The  matching  algorithm  must  be 
based  upon  individual  matches  between  the  image’s  extracted  edges  and  the  model’s  linear  features.  When 
comparing  two-dimensional  line  segments,  three  aspects  must  be  considered:  translation,  rotation,  and  scaling 
differences  (Figures  9.1  through  9.3). 


Figure  9.1.  The  translation  distance,  d,  between  two  lines. 
Can  be  computed  as  the  median  between  distances  dl  and 
d2 ,  perpendicular  to  the  model  line. 


Figure  9.2.  The  rotational  difference,  cc  between  two  lines. 
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Image  line 


modal  line 


scaling  factor,  a  ■ 


length  of  image  line  mapped  onto  model  line 
length  of  model  line 


Figure  9.3.  The  scaling  factor,  s,  between  two  lines.  S  is  die  ratio  of  the  image  line 
mapped  onto  the  model  line  with  the  model  line  itself. 


An  additional  requirement  based  upon  the  properties  of  the  extracted  edges  using  least  squares  fit  can 
also  be  stipulated.  That  is,  the  endpoints  of  the  image  edges  must  lie  within  the  endpoints  of  die  modelled  line 
segment.  This  check  for  endpoint  inclusion  may  be  relaxed  to  allow  for  vehicle  translation  errors.  This  is  done 
by  keeping  the  difference  that  the  image  edge  endpoints  lie  outside  of  the  endpoints  of  the  modelled  line 
segment,  divided  by  the  length  of  the  image  edge  (rffflajor),  less  than  or  equal  to  a  specified  amount  (k  in  Figure 
9.4). 


model  line 


Figure  9.4.  The  inclusion  of  image  endpoints  within  the  model  line’s 
endpoints.  Image  line  2  can  meet  this  requirement  if  (A/ 1)<  k,  (0.0  <  k<  0.1). 


Another  consideration  for  matching  image  lines  to  model  lines  is  that  since  the  extracted  image  lines 
are  often  incomplete  and  broken  segments  of  a  modelled  feature,  many  image  lines  may  be  matched  to  one 
model  line  as  shown  in  Figure  9.5.  The  only  stipulation  is  that  the  image  lines  do  not  overlap  when  mapped 
onto  the  same  model  line. 
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Image  line  1 


Image  line  3 


Figure  9.5.  Many-to-one  relationship  of  image  lines  to  model  lines. 
Image  lines  1, 2,  and  3  may  all  be  matched  with  the  model  line. 


These  five  aspects  of  two-dimensional  line  matching  could  be  combined  to  produce  one  value  to 
describe  the  confidence  for  the  match  of  one  image  line  to  one  model  line.  Summing  individual  matches  with 
highest  confidences  provides  the  most  likely  matching  between  image  edges  and  model  line  segments,  yet  it 
remains  to  be  determined  if  it  is  the  correct  matching.  Beveridge  [BEV90]  includes  an  error  term  for  model 
line  segments  that  are  omitted  from  the  matching  process.  Many  graph-based  matching  algorithms  outlined 
by  Ballard  [BAL82]  are  NP-complete  problems.  Beveridge’s  method  defaults  to  continually  permuting 
various  matching  combinations  in  order  to  find  the  optimum  matching. 


C.  MATCHING  IMPLEMENTATION  FOR  VERTICAL  LINE  SEGMENTS 

Assuming  that  the  robot’s  camera  is  in  a  fixed  position  and  that  the  robot  is  on  a  stable  platform,  the 
pose  (or  configuration)  of  the  robot  can  be  described  in  three  degrees  of  freedom  (x0,y0,%).  With  this 
assumption,  all  vertical  lines  in  the  real  world  will  be  vertical  in  any  image.  Therefore,  matching  only  vertical 
lines  should  provide  a  simple  solution  for  the  pose  determination/correctioo  problem.  Since  all  line  segments 
will  be  vertical,  rotational  differences  between  the  image  lines  and  model  lines  may  be  neglected,  and  the 
translational  distance,  d,  can  be  simplified  to  be  the  horizontal  angle  from  the  midpoint  of  the  image  line  to 
the  model  line 

Considering  an  image  plane  M  that  exists  at  focal  length  /  from  the  center  of  the  lens  system  of  the 
robot ’s  camera,  the  vertical  edges  extracted  from  the  image  will  appear  as  in  Figure  9.6.  For  a  given  estimated 
pose  [\„.y0.%).  an  overview  of  the  robot  and  the  image  plane  M  (Figure  9.6)  will  be  situated  in  the  operating 
environment  model  as  in  Figure  9.7.  The  environment  floor  is  the  vv  plane  and  heights  of  modelled  features 
are  expressed  in  the  r-axis.  Image  plane  M  is  parallel  with  the  r-axis. 
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image  plane  M 


u 


Figure  9.6.  The  image  plane  M.  The  image  is  centered  on  image  axes  u,v  with  three 
extracted  vertical  lines  at  u2.  u2,  and  u}. 


Figure  9.7.  The  robot  in  the  environment  coordinate  system.  The  robot’s  pose  is  defined  by  (xw  y0,%). 
The  image  plane  M  exists  at  the  focal  length  (f)  from  the  robot’s  camera.  Vertical  lines  extracted  from 
the  input  image  in  Figure  9.6  (at  image  u-coordinate  positions  uj,  u2,  and  u2)  can  be  described  by  an 
angle  a  from  the  center  of  the  image. 
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A  two-dimensional  view  of  tbe  environment  model  is  constructed  given  the  three-dimensional  model, 
an  assumed  pose,  and  the  focal  length  /  (Figure  9.8).  Tbe  assumed  pose  is  provided  by  the  robot’s  dead 
reckoning  capabilities  from  wheel  motion  and  is  used  knowing  that  it  is  an  approximation.  Therefore ,  the 
matching  problem  to  evaluate  how  well  the  image  edges  from  the  image  plane  M  fit  onto  the  model  lines  in 
the  two-dimensional  plane  M’.  For  this  implementation  using  only  vertical  lines,  quantifying  tbe  match 
between  an  edge  and  a  model  line  will  be  based  only  upon  the  translational  distance  d,  if  the  endpoint 
inclusion  requirement  is  met. 


figure  9.8.  The  image  plane  M' .  M'  is  tbe  two-dimensional  view  of  tbe  modelled  environment 
for  the  given  pose  U0,v0,90)  of  the  robot  (camera).  M '  can  be  thought  to  exist  at  a  distance  equal 
to  the  focal  length,/,  of  the  camera’s  lens  system. 

The  line  matching  provides  connectivity  between  the  extracted  image  lines  and  the  vertical  features  of 
the  environment  model.  Thus,  the  three  most  significant  extracted  image  lines  can  be  matched  with  three 
vertical  model  lines  as  in  Figure  9.9.  Vertical  image  lines  uj.  u2.  and  uj,  from  tbe  image  plane  M,  are  best 
matched  with  vertical  model  features  LI ,  L2,  and  L3,  respectively  This  does  not  ensure  that  this  is  the  correct 
matching,  but  it  will  provide  a  means  to  calculate  a  possible  pose  for  the  robot.  This  method  also  assumes  that 
the  three  most  significant  vertical  image  lines  are  produced  from  modelled  linear  features 
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Since  backtracking  may  be  necessary  for  recovering  from  successive  attempts  in  the  pose 
determination  algorithm,  every  image  edge  in  M  must  be  able  to  suppoit  a  linked  list  to  all  of  the  possible 
matches  to  the  model  line  segments  in  M'.  Appendix  C  contains  the  code  for  this  implementation,  titled 
vertmatch.  A  data  structure  named  MATCHTYPE  in  the  file  match_types.h  (Appendix  A)  is  used  for  the 
association  of  an  extracted  edge  with  a  model  line  segment  Each  image  edge  data  structure  has  a  pointer  to 
the  head  of  the  list  of  possible  MATCHTYPEs  and  a  separate  pointer  for  the  best.  The  different  possible 
matches  of  an  image  edge  to  various  model  line  segments  are  ordered  by  horizontal  angular  difference 
between  the  location  of  image  edge  and  model  line.  The  smallest  difference,  that  being  the  best  match,  is  at 
the  head  of  the  match  list  and  all  subsequent  matches  follow.  The  pose  determination/correction  algorithm  in 
Chapter  X  utilizes  only  the  three  most  significant  extracted  image  edges  (i.e.  the  three  image  lines  with  the 
greatest  values  for  Thus  the  vertmatch  implementation  creates  a  list  of  MATHCTYPEs  for  the  three 

most  significant  edges. 


Figure  9.9.  Modelled  vertical  features  matched  with  extracted  vertical  edges  from  an  image.  LI,  L2, 
and  L3  are  the  labels  assigned  to  the  vertical  lines  matched  to  the  three  most  significant  vertical 
image  lines  in  found  in  M.  Angle  y  is  the  orientation  of  the  line  from  the  camera’s  position  to  the 
location  of  a  vertical  line.  The  modelled  vertical  line  should  then  appear  at  an  angle  (0O  -  y)  from 
the  center  of  plane  M’,  the  2D  view  of  the  model  environment  (Figure  9.8). 
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X.  CORRECTING  ROBOT’S  POSE 


With  the  assumption  that  the  robotic  vehicle  is  a  stable  platform  with  it*  era  at  a  fixed  position, 
there  are  only  three  degrees  of  freedom  in  the  model  space  (x,  y,  and  6  -  corresponding  to  the  variables  of  the 
robot's  pose)  that  need  to  be  determined.  If  considering  only  vertical  lines  for  matching,  it  is  possible  to 
determine  the  camera’s  pose  from  only  the  three  most  significant  vertical  lines  in  an  image,  provided  that  they 
are  all  produced  from  features  represented  in  the  three-dimensional  environment  model. 


A.  POSE  DETERMINATION 


The  only  known,  accurately  measured  quantities  that  are  available  for  pose  determination  are  the  .rv 
coordinate  map  locations  of  the  three  vertical  model  lines  and  the  horizontal  angular  differences  between  the 
extracted  vertical  edges  uj,  u2,  and  uj  from  Figure  9.6.  Let  us  denote  two  of  these  angles,  the  angle  between 
the  leftmost  (uj)  and  middle  (u2)  image  lines  and  the  angle  between  the  middle  (u2)  and  rightmost  (u2)  image 
lines  as  a  and  p.  respectively.  If  the  best  matches  for  u2.  u2,  and  uj  are  used  (vertical  model  lines  LI,  L2,  and 
L3),  angle  a  is  the  expected  viewing  angle  between  LI  and  L2,  and  P  is  the  expected  viewing  angle  between 
L2  and  L3.  There  is  exactly  one  location  that  will  have  viewing  angles  a  and  P  between  model  lines  LI,  L2, 
and  L3.  If  the  distance  between  two  vertical  model  features  is  thought  of  as  the  side  of  a  triangle  (a)  and  the 
associated  viewing  angle  (a)  as  the  opposite  angle  of  the  triangle,  an  infinite  number  of  possible  locations  for 
the  viewing  angle  exist  on  a  circle  that  circumscribes  all  possible  triangles  (Figure  10.1).  For  two  triangles, 
one  with  side  a  and  opposite  angle  a,  and  the  other  with  side  b  and  angle  p,  two  circles  of  possible  viewing 
locations  can  be  constructed  as  in  Figure  10.2.  Sides  a  and  b  are  therefore  chords  of  the  two  circles.  There  are 
two  intersections  of  the  circles,  provided  that  the  circles  are  not  identical.  One  intersection  is  at  the  location 
of  vertical  model  line  L2  and  the  other  is  the  possible  position  of  the  camera  (xp0OT,  y^).  The  camera  must 
be  at  this  position  for  this  line  matching,  given  the  viewing  angles  a  and  p  between  the  three  major  vertical 
image  edges  in  M. 


Figure  10.1.  Triangles  with  side  a  and  opposite  angle  a.  circumscribed  by  a  circle 
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Figure  10.2.  Geometry  for  pose  determination.  Given  measured  locations  of  vertical  features 
in  the  environment  model  (Ll,  L2,  and  L3),  circle  chords  a  and  b  can  be  calculated.  Angles 
a  and  P  are  measured  from  the  image  plane  and  are  defined  by  the  angular  difference  between 
the  vertical  extracted  edge  features  from  the  image.  With  chords  a  and  b  and  angles  a  and  p,  two 
circles  can  be  constructed  such  that  their  intersections  will  exist  at  file  position  of  model  line  L2 
(-VL2'-Vu)  ^  a  possible  robot  position  (jr^.y^). 


Once  a  position  has  been  determined,  calculation  of  the  camera’s  orientation  in  die  environment  is 
simple.  Since  the  locations  of  the  vertical  modelled  features  are  known  in  the  environment  map  (xy,  yy),  the 
angular  orientation,  y.  from  the  possible  position  of  the  camera  (Xpc,  is  atan2(xu  -  x?om,  yy  -  Vpo«)- 
The  angular  location  of  these  vertical  model  lines  from  the  vertical  centerline  axis  in  the  two-dimensional 
plane  M'  is  0C  -  y  Averaging  the  differences  between  angles  of  the  model  lines  from  the  center  of  the  plane 
M'  (6C  -  y)  and  the  angles  of  the  image  lines  from  the  center  of  the  image  plane  M  provides  a  rotational 
correction.  The  correction  is  added  to  the  orientation  of  the  input  (dead-reckoning)  pose.  %,  to  get  the 
possible  pose  orientation,  9^.  The  pose  for  the  camera  based  upon  the  given  matching  is  therefore 
determined  (x^.  v^,  9^). 
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B.  POSE  VERIFICATION 


The  next  best  match  combination  will  be  performed  by  selecting  the  next  match  with  the  least 
horizontal  angle  difference  between  the  vertical  image  edge  and  model  line  segment  out  of  all  three  image 
line  match  lists.  For  this  new  combination  of  three  matches,  another  possible  pose  is  calculated  by  the  above 
method.  This  is  done  for  the  n  best  match  combinations.  An  exception  to  this  rule  is  to  drip  matches  where 
two  or  more  image  lines  are  matched  to  the  model  line  segment.  This  method  will  provide,  at  most,  n  possible 
poses  from  the  n  match  combinations. 

Since  a  unique  pose  can  be  determined  from  any  given  matching  combination  for  the  three  most 
significant  image  edges,  the  determination  of  the  correct  pose  of  the  robot  must  rely  upon  a  means  to  verify 
the  pose  and  the  matching  combination.  This  topic  is  described  by  Heller  and  Stenstrom  [HEL89].  For  a  given 
possible  pose  y  ,  0^,),  locations  for  all  the  model  line  segments  can  be  calculated  if  they  were  to 
be  viewed  in  the  image  plane  M.  Comparing  all  of  the  extracted  image  lines  to  see  how  many  fit  the 
repositioned  model  lines  provides  a  means  to  evaluate  the  possible  pose.  However,  since  many  image  lines 
are  not  produced  from  modelled  features  using  the  number  of  good  image  to  model  line  fittings  alone  can  not 
guarantee  the  correct  pose. 

If  some  credibility  is  expected  of  the  estimated  pose  for  which  the  two-dimensional  view  of  the  model 
(A/'  jwas  constructed  from,  then  calculated  possible  poses  should  be  close  to  the  estimated  pose.  Thus,  an 
evaluation  (evalposs)  combining  the  number  of  image  edge  to  model  line  segment  fittings  (m^)  with  the 
distance  between  the  possible  pose  and  the  estimated  pose  (dposs)  will  be  of  the  form 

(Eq  10.1) 

uposs 

The  possible  pose  with  the  greatest  value  for  will  then  be  considered  the  corrected  pose. 

Results  of  this  method  are  shown  in  Chapter  XI. 
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XL  PATTERN  MATCHING  AND  POSE  DETERMINATION  RESULTS 


The  results  from  three  trials  are  shown  in  this  chapter.  The  actual  camera  pose  is  measured  from  a 
reference  point  in  the  operating  environment  The  estimated  pose  for  developing  the  two-dimensional  view 
of  the  model  environment  was  encoded  in  the  program  vertmatch  to  simulate  dead-reckoning  errors.  Each 
trial  has  a  series  of  four  figures.  The  first  figure  is  the  input  image  with  the  extracted  vertical  edges.  The 
second  Figure  is  a  wire  frame  two-dimensional  view  of  the  environment  model  based  upon  the  estimated  pose. 
The  third  figure  is  the  wire-frame  model  superimposed  over  the  input  image.  The  fourth  figure  is  a  wire-frame 
model  based  upon  the  corrected  pose  superimposed  over  the  image.  The  inputs  to  the  vertmatch  routine  are 
the  image,  the  environment  model,  and  the  estimated  pose.  Output  from  the  vertmatch  routine  listing  the 
matches  and  possible  poses  is  shown  between  the  third  and  fourth  figures  of  each  trial. 


A.  TRIAL  1 


Figure  11.1.  Trial  1,  input  image  with  extracted  edges. 

Actual  camera  pose:  xq  *  60.0  inches,  y0  -  366.0  inches,  0q  *  253. (f. 
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Figure  1 1.2.  Trial  1, 2D  wire-frame  view  of  model  based  upon  estimated  pose. 
Estimated  pose:  xq  =  60.0  inches,  yo  =  366.0  inches,  0O  =  250.0°. 


Figure  11.3.  Trial  1, 2D  wire-frame  view  superimposed  over  input  image. 


Script  started  on  Thu  Mar  19  12:12:18  1992 
turing  1%  vertmatch  060366253 .pic 

vertmatch>  060366253 .pic  xsize=  646  ysize=  486  pixels=  313956 
lines  found  in  image  written  to:  ‘lines. text' 

Number  lines  found  in  060366253 .pic  =  17 

DR  (input)  pose:  x  =  60.00,  y  =  366.00,  theta  =  250.00(-1.92  rads) 

Nr  Vertical  Model  Lines  =  20 

Determine  3  major  image  lines  for  matching:  - 

1)  line  13,  angle  from  center  =  -0.1819 

2)  line  10,  angle  from  center  =  0.0980 

3)  line  8,  angle  from  center  =  -0.0814 

Image  line  matchlists  to  Model  line  NAME (angle  difference):  - 

left  (edge  10)  > 

N  (0 . 0526 )  M (-0 .0634 )  L(-0.0675)  I(-0.0904>  J(-0.0904)  H(-0.1279) 

G  (-0 . 132 1 )  F (-0 . 1737)  E(-0.2176)  D(-0.2224)  C(-0.2288)  A(-0.2300) 

middle  (edge  8)  > 

F (0 . 0057 )  E  (-0 . 0382 )  D(-0.0430)  G(0.0473)  C(-0.0494)  A(-0.0506) 

H (0 . 0515 )  J (0.0890)  1(0.0890)  L(0.1119)  M(0.1160)  N(0.2320) 

right  (edge  13)  > 

A (0 . 0498)  C  (0 . 05 1 1 )  D(0.0575)  E(0.0623)  F(0.1062)  G(0.1478) 

H  (0 . 1 520 )  J (0 . 1895)  N(0.3325) 

Pose  determination:  - 

l  N,  F,  A]  x=  55.42  y=  513.24  T=  147.3124  R=  -0.3730  nr  verify  matches=  5 
***  BEST  POSE  MODIFIED  *** 

[  N,  E,  A]  x=  276.04  y=  256.66  T=  242.1306  R=  -0.6891  nr  verify  matches=  0 
[  N,  D,  A]  NO  position:  chord  length  <  MIN_CHORD_LENGTH 

(  N,  G,  A]  x=  51.84  y=  381.92  T=  17.8955  R=  0.0048  nr  verify  matches=  6 

***  BEST  POSE  MODIFIED  *** 

[  N,  C,  A]  NO  position:  chord  length  <  MIN_CHORD_LENGTH 
[  N,  A,  A]  NO  position:  2  IMG_LINEs  matched  to  same  model  LINE 
[  N,  A,  C]  NO  position:  chord  length  <  MIN_CHORD_LENGTH 

[  N,  H,  C]  x=  58.49  y=  368.04  T=  2.5417  R=  0.0464  nr  verify  matches=  7 

***  BEST  POSE  MODIFIED  *** 

[  N,  H,  D]  x=  58.48  y=  379.09  T=  13.1814  R=  0.0087  nr  verify  matches=  7 

[  N,  H,  E]  x=  86.10  y=  459.31  T=  96.8872  R=  -0.2843  nr  verify  matches=  6 

[  M,  H,  E]  x—  307.54  y=  268.49  T=  266.0523  R=  -1.1971  nr  verify  matches=  0 

(  L,  H,  E]  x—  307.41  y=  268.42  T=  265.9568  R=  -1.1694  nr  verify  matches=  0 

[  L,  J,  E]  NO  position:  chord  length  <  MIN_CHORD_LENGTH 

[  L,  I,  E]  NO  position:  chord  length  <  MIN_CHORD_LENGTH 

[  I,  I,  E]  NO  position:  2  IMG_LINEs  matched  to  same  model  LINE 
[  J,  I,  E]  NO  position:  chord  length  <  MIN_CHORD_LENGTH 

(  J,  I,  F]  NO  position:  chord  length  <  MIN_CHORD_LENGTH 

[  J,  L,  F]  NO  position:  chord  length  <  MIN_CHORD_LENGTH 

(  J,  M,  F]  NO  position:  chord  length  <  MIN_CHORD_LENGTH 

[  H,  M,  F]  x=  276.86  y=  247.81  T=  246.9754  R=  0.8255  nr  verify  matches=  2 

[  G,  M,  F]  x=  270.77  y=  246.45  T=  242.3155  R=  0.8120  nr  verify  matches=  2 

(  G,  M,  G]  NO  position:  2  IMG_LINEs  matched  to  same  model  LINE 
(  G,  M,  H]  x=  307.93  y=  268.25  T=  266.5063  R=  -1.4201  nr  verify  matches=  0 

(  F,  M,  H]  x=  340.40  y=  274.86  T=  294 .8414  R=  -1.5850  nr  verify  matches=  0 
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(  F,  M,  J]  NO  position:  chord  length  <  MIN_CHORD_LENGTH 

(  E,  M,  J]  NO  position:  chord  length  <  MIN_CHORD_LENGTH 

[  D,  M,  J]  NO  position:  chord  length  <  MIN_CHORD_LENGTH 

[  C,  M,  J]  NO  position:  chord  length  <  MIN_CHORD_LENGTH 

[  A,  M,  J]  NO  position:  chord  length  <  MIN_CHORD_LENGTH 

[  A,  N,  J]  x=  337.19  y=  307.83  T=  283.2245  R=  -0.5412  nr  verify  matches 
[  A,  N,  N]  NO  position:  2  IMG_LINEs  matched  to  same  model  LINE 


Corrected  pose:  x=  58.49,  y=  368.04,  theta=  252 . 66 (-1 . 8735  rads) 
turing  2%  exit 
turing  3% 

script  done  on  Thu  Mar  19  12:13:35  1992 


Figure  1 1.4.  Trial  1,  corrected  2D  wire-frame  view  superimposed  over  input  image. 
Corrected  pose:  xq  =  58.49  inches,  yo  =  368.04  inches,  9q  =  252.66°. 

Translational  error  from  actual  pose  =  2.54  inches,  rotational  error  =  0.34°. 
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TRIAL  2 


Figure  1 1 .5.  Trial  2,  input  image  with  extracted  edges. 

Actual  camera  pose:  x0  =  59.0  inches,  y0  =  366.0  inches,  0O  =  249.0°. 


61 


Figure  1 1.6.  Trial  2,  2D  wire-frame  \iew  of  model  based  upon  estimated  pose. 
Estimated  pose:  \0  =  60.0  inches,  y0  =  366.0  inches,  By  =  250.0°. 


Figure  1 1 .7.  Trial  2,  2D  wire-frame  view  superimposed  over  input  image. 
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Scrip-  s’.- a r i c o  o r ] nu  Ma r  19  I2;19i42  -992 
turirg  1%  ver:ra'.cr  05  936624  9  .  pic 

ver;'3'.cr.>  059366249 .cic  xsize=  6  ^  6  ysize*  486  pixels*  313956 
lir.es  fcs.no  ;r.  image  written  to:  'iir.es .  text' 

Number  tires  rc.r-.o  in  05936624 9 . pic  =  34 

DR  (ir.r-t )  peso:  x  -  60. CO,  y  =  366.00,  theta  =  250. CO  (-1.92  rads) 


Nr  Vert i ca.  Mcce.  lines  =  20 

Deter-. ne  3  -ait-  image  lines  for  matching: - 

1)  ..re  2'.,  ar.g.  e  from  center  =  -0.1113 

2)  lire  2 1 ,  arr.e  from,  center  =  G.1663 

3)  13,  arc.-:.  from  center  -  C.05C9 

- -  - ~  ^  -  •  "  s - s  wC  Xc3£  iir^s  KAMI  (cr.sl 6  ci550r cr-cc)^  — —  —  —  — 

_e!  -  ^ 

N  ( -  .  1 .51 1  )  K-C.1358)  1  ( -  0  . 1 1 6 1  )  J(-3.1587)  H  (-C  .  1  962  ) 


-  ..?•■;  0)  -  (-C.2855)  D(-0.25t  )  *0  (-0.2971)  AI-0.2983) 


!_<:)  :  (-0.0453  )  J  ( -0  .04  33)  Hl-C.0808)  G  < -C  .  0  850 ) 

•16  !  £(-0.1  ’CO  0  ( -  0  .  1  ’  :  3  i  01-0.181')  A  ( -C  .  1829) 


..-0)  C  ( -C  .  0 1  94  )  A  ( -  0  .  0  2  0  )  1(  0  .  0  357)  C-(0.0773) 
]  1 (0.1150)  1(0.1419)  y (0.1460)  NIC. 2619) 


..3813  --  4  r.r  verify  matches*  10 

-.39  =  .'  -  -  r.r  verify  matches*  17 

.  .  -4  :  5-  -1.1.  1  r.r  verify  matches*  16 

2.74  9s  ?  =  -0.04  96  r.r  verify  matches*  15 

2.3339  -0.144s  rr  verify  matches*  15 

,39.bci'  t  -....39  r.r  verify  matches* 

v-  4  56..-"  1-  151  .cl  cl  ••  -0.7514  rr  verify  matches*  9 

4  c  S  . 1 1  91.60  61  1-  -..'514  rr  verify  matches*  9 

/  •  446.22  7=  206. 9:1  1-  -..8209  rr  verify  matches*  8 

isr. :  chore  ier.gtr.  <  yi!._CH0~.O__ENGTH 
c*".  i  2  MG  L  I  N  E  s  t  a c  r.  0  C2  s  d  t  0  to  G  0  1  LINE 

err  rc  .er.gtr  ■  M:>._C.-:0?1._1E\GTH 
ic:  :  z  1X0  i  IMs  ~stcr.ec  t.  sa— e  mcce.  11NE 

Isr.:  2  1  yo  1  IMs  -ttcr.ec  ts  same  mode.  11NE 

.or.:  2  .MG  11  Ms  matcr.ec  ts  same  mcaei  LINE 

y-  213.99  7'  252.6534  9-  1.0803  r.r  verify  matches*  7 

.-  259.29  75  2'4. 5149  R  -0.334  8  rr  verify  matches*  C 

y  ~  2  58. .0  7*  2  2  .  s  3  3  2  3*  -  0  .  3  2  3  0  r.r  verify  matches*  0 

ter. :  2  1 XG  HMs  rrstcr.es  ts  same  model  LINE 
y-  259.25  7*  272.3296  R-  -3.2428  rr  verify  matches*  C 

y^  257.00  7*  270.1  404  ?.*  -0.3378  r.r  verify  matches*  0 

y  257.70  7*  270.14  14  R  -  0.3379  r.r  verify  matches*  C 

y  257.1  5  7-  249.1  "2  ••  -11.4911  rr  ver.fy  matches*  C 
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J,  I  , 

X  ' 

2.  *L  *  .  .9 

y-  2  6"?  .  95  1=  223 .8134  R=  -C. 

6992 

r  r  verify 

matches*  C 

z ,  z ,  x 

x  - 

2  5::  .  5-5 

y=  267.95  T =  221 .4 V89  R=  -C . 

7003 

r.r  verify 

matches*  0 

a,  y. 

X- 

.  e: 

y-  267.68  T=  221.6C25  Ft  =  -0 . 

6970 

r.r  verify 

marches5  0 

X  - 

.  :.  .  -  ; 

y-  267.44  7=  220.6821  R=  -C. 

7016 

r.r  verify 

matches*  0 

3,  a,  y 

x^ 

..  C  .  J 

y-  267.42  7=  220.5905  R-  -0. 

7022 

r.r  verify 

matches*  0 

*  /  r-t  y 

X- 

.  . =4 

y-  266. 64  7=  207.8188  R-  -0. 

8255 

r,r  verify 

matches5  0 

-  t  r\ ,  ’ . 

X 

2  .  .  5 

y  267.20  7=  216.0:5;  R-  -1 . 

£.“?  I  7 

r.r  verify 

matches*  0 

,  r\,  7 

X 

.  :  .*'9 

y  2  66  .  c  4  7  -  .  . 

264  : 

rr  verify 

matches5  0 

t  r'  t 

Z  y  .  ■  .L  -  w 

i  c. r  :  c r.cro  ^ ergt. r.  <  XI N  Zr. Z 

R5  EE 

NGTH 

z 

:  .  s :  i 

i cr. :  chord  ler.gtr.  <  XI N  CHC 

R0  LENGTH 

A ,  A ,  ' 

,cr. :  2  IXG^LINEs  .Tsicnec  ;c 

Sa."i6 

~odei  LI 

NE 

r;.-. 

'  ; 

H  .  ' ,  y  -•  3  6  c  .  4  z. ,  i : .  *  *w  a  =  2  4  9 

.02  1- 

1 . 93 7 C  ra 

as) 

-  y.sr  19  72:2C::>6  1992 


Figure  1 1.8.  Trial  2,  corrected  2D  wire-frame  view  superimposed  over  input  image. 
Corrected  pose:  \0  =  58.73  inches,  yo  =  366.45  inches,  0O  =  249.02°. 

Translational  error  from  actual  pose  =  1.35  inches,  rotational  error  =  0.02°. 
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TRIAL  3 


Figure  1 1 .9.  Trial  3,  input  image  with  extracted  edges. 

Actual  camera  pose:  \,  =  43.0  inches,  y0  =  277.0  inches,  0O  =  356.0°. 


Figure  11.10.  Trial  2.  2D  wire-frame  view  of  model  based  upon  estimated  pose. 
Estimated  pose:  ,\„  =  4S.0  inches,  y0  =  277.0  inches.  0()  =  000.0°. 
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Figure  11.11.  Trial  3,  2D  wire-frame  view  superimposed  over  input  image. 
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Figure  11.12.  Trial  3,  corrected  2D  wire-frame  view  superimposed  over  input  image. 
Corrected  pose:  x0  =  46.55  inches,  y0  =  260.39  inches,  0U  =  356.42°. 

Translational  error  from  actual  pose  =  16.67  inches,  rotational  error  =  0.42°. 
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xn.  CONCLUSIONS 


A.  FEATURE  EXTRACTION 

The  edge  extraction  implementation  fastedge,  provides  linear  features  that,  unlike  the  Hough 
transform,  have  good  endpoint  information.  The  implementation  requires  approximately  IS  seconds  to 
extract  the  linear  edges  from  a  486  x  646  pixel  image  (313,956  pixels  total)  on  the  Personal  Iris  (35  MHz 
clock  cycle).  For  a  picture  that  has  been  “shrunken”  to  half  of  its  width  and  height,  the  process  requires  only 

3  seconds. 

Selecting  the  appropriate  constants  C,  through  C,  is  important.  The  effects  of  modifying  these 
parameters,  outlined  in  Chapter  VII,  provide  optimum  values  for  use  in  a  specific  environment.  The  task  of 
determining  optimum  values  for  a  dynamic  environment  could  be  automated;  however,  it  would  require  an 
extra  scan  through  the  image  or  else  added  hardware  (e.g.  a  lightmeter). 

The  version  implementing  two  static  sets  of  angles  used  for  the  description  of  pixel  gradient 
orientations  could  provide  a  very  fast  hardware  implementation.  Although  it  did  not  perform  as  well  as 
averaging  gradient  orientations,  the  speedup  advantages  gained  from  implementing  this  method  warrant  more 
consideration  and  experimentation. 

B.  PATTERN  MATCHING 

Pattern  matching  is  the  “weak  link”  in  the  vertmatch  implementation.  Although  the  method  is  quick, 
it  is  crude.  It  was  chosen  as  a  means  to  provide  a  simple  basis  for  the  pose  determination  problem,  therefore 
opting  for  the  easier  problem  of  matching  only  two-dimensional  vertical  line  segments  with  the  assumption 
that  the  three  most  significant  edges  extracted  from  the  image  are  products  of  vertical  modelled  features. 
Many  lii  ti. aching  implementations  are  graph  based  and  therefore  rely  upon  links  between  the  extracted  line 
segments  that  can  only  be  inferred,  but  not  accurately  determined.  Graph  based  matching  algorithms  are  often 
NP  complete  thus  leading  to  alternative  methods  like  Beveridge’s  randomly  permuting  matches  to  find  the 
optimum  match. 

C.  POSE  DETERMINATION 

The  pose  determination  algorithm  was  based  upon  simple  vertical  line  matching  in  a  known  orthogonal 
environment  and  the  geometry  for  its  foundation  allows  a  unique  pose  to  be  calculated  simply  and  quickly.  It 
was  found  to  provide  good  estimates  from  three  visual  lines  of  bearings.  Small  errors  between  the  actual  and 
corrected  poses  in  the  three  trials  in  Chapter  XI  could  be  from  the  algorithm,  or  from  hardware  floating-point 
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constraints  involved  with  the  trigonometric  functions,  or  (most  likely)  from  data  measurement  error  from  the 
tape  measure. 

The  pose  determination  test  trials  were  conducted  at  two  different  primary  locations,  viewing  the 
elevator  alcove  and  viewing  the  long  axis  of  the  hallway.  Trials  1  and  2  in  Chapter  XI  were  conducted  with 
the  environment  features  of  the  elevator  alcove  approximately  20  feet  from  the  camera  which  provided  only 
20  vertical  model  line  segments.  With  relatively  few  model  line  segments  for  the  three  major  image  edges  to 
be  compared  with,  the  matching  was  easy  to  check.  Trial  3  involved  much  greater  distances  and  over  100 
vertical  model  line  segments.  The  corrected  pose  could  be  more  accurate  if  more  iterations  for  possible 
matches  and  poses  are  conducted.  However,  the  actual  pose  can  be  determined  if  all  match  combinations  are 
tested  exhaustively.  Implementation  of  backtracking  with  some  good  heuristics  about  the  matching  process 
would  be  beneficial  here.  The  long  aspect  of  the  hallway  for  pose  determination  is  more  difficult  in  terms  of 
distances  and  possible  matching  combinations;  however,  other  features  (edges  from  baseboards  and  overhead 
fights)  may  be  utilized  to  enhance  the  performance.  Thus,  a  more  general  and  more  robust  line  matching 
implementation  is  required. 

D.  FOLLOW-ON  WORK  FOR  YAMABICO-11 

The  methods  developed  in  this  thesis  only  have  been  directed  towards  the  first  objective  of  visual 
navigation  in  a  known  environment  and  has  not  dealt  with  the  greater  complexities  of  object  recognition  in 
an  unknown  environment.  There  remaias  much  work  to  be  done  for  continuing  the  implementation  of  visual 
navigation  capabilities  on  Yamabico-1 1.  Some  of  these  areas  are; 

Coding  the  methods  in  this  thesis  in  68020  microprocessor  assembly  language. 

Developing  a  vision  based  support  programming  language  similar  to  the  Mobile  Motion 
Language. 

Implementing  the  environment  model  support  routines  in  hardware. 

Developing  a  multiprocessor  architecture  to  handle  image  edge  extraction,  two-dimensional 
environment  view  construction,  line  matching,  and  pose  determination  in  parallel. 

Incorporating  sensor  fusion  of  sonar  data  with  the  vision  navigation  routines  to  check  accuracy. 

Develop  and  implement  fast.  robust  line  matching  method  for  all  lines. 

Develop  a  method  to  positively  determine  if  an  edge  is  from  a  modelled  linear  feature. 

Develop  object-based  recognition  methods  to  handle  known  and  unknown  objects. 

Develop  an  vision  based  algorithm  that  supports  a  reflexive  behavior  for  obstacle  avoidance. 

Develop  methods  to  track  moving  obstacles. 
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APPENDIX  A  -  DATATYPES 


A.  FUNCTION:  AT  AN  2 

/**«**••*•*******•*«•*»****«********»**«•*••*******•••*»•****•»** •t****** 

P  file:  aian2.c 
P 

P  Provide  a  definition  of  the  atan2()  function. 

/* 

f*********>ni***m******mm*************m***9********»*»*************tim****i 

#define  PI  3.141592653589793 


double  atan2(y,x) 
double  x,  y; 

if  (x  >  0.0)  return  (arctan(y/x)); 

else  if  ((x  <  0.0)  &  Sc  (y  >  0.0))  return  (arctan(y/x)  +  PI); 

else  if  ((x  <  0.0)  &&  (y  <  0.0))  return  (arctan(y/x)  -  PI); 

else  if  ((x  <  0.0)  &&  (y  ==  0.0))  return  (PI); 

else  if  ((x  ==  0.0)  &&  (y  >  0.0))  return  (PI/2.0); 

else  if  ((x  ==  0.0)  &&(>-<  0.0))  return  ( -PI/2.0); 

else  return  (0.0); 

) 
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B.  DATA  TYPE  HEADER  FILE:  IMAGE  TYPES.H 

/*********************************¥*****%**] ,c***************************** 

/*  file:  image_types.h 

/* 

/*  This  file  holds  the  structure  definitions  for  the  different 
/*  types  of  in-memory  images. 

/*  Types  defined:  CMAP1MAGE 
/*  NPSIMAGE 

tfdefine  RGBA  1  /’  RGB  A  24  bit  images  (alpha  is  Oxff  filled)  *! 

^define  CMAPPED  2  /’  color  mapped  images  *1 

#define  RGBAW1THALPHA  3  /*  RGBA  32  bit  images  where  alpha  is  read/saved 
in  the  image  files.  */ 

/*  define  a  structure  type  for  color  mapped  images  */ 

struct  emapimage 

{ 

short  ’bilsptr;  /*  the  bits  for  the  short  images  ’/ 
long  nentries:  /*  the  total  number  of  e^iries  in  the  color  map  ’/ 
short  Teds;  /*  ptr  to  the  red  entries  of  the  color  map  */ 
short  ’’greens:  /’  ptr  to  the  green  entries  in  the  color  map  ’/ 
short  ’blues;  /*  ptr  to  the  blue  entries  in  the  color  map  */ 

lone  cmapoffset:  /’  color  map  offset,  i.e.  the  first  color  we  w  ill  us  in  the  color  map  */ 

}: 

typedel  struct  cinupunege  CM  API  MAGE:  ■'*  deline  a  CMAP1MAGE  type  */ 


/*  define  a  union  so  that  the  lop  level  image  structure’s  last  pointer  can  point  to  several  different 
kinds  of  images.’; 

union  imaged  ptr 

1 

long  ’bitspir:  /*  long  images  need  no  more  data  than  a  ptr  to  the  bus.  */ 

CM  API  MAGE  ’  cn.upi ”  a  color  mappcJ  image  must  have  the  bits  and  a  color  map  so 

we  need  a  v.omplete  structure.  */ 


r  define  the  top  level  structure  for  the  image  */ 

struct  image 

( 

long  type:  /’  image  type  T 

long  xsi/e:  r  \si/e  of  die  image  */ 

long  ysi/.e:  /’  vsi/e  of  the  image  */ 

char  ’name;  /’  ptr  to  string  naming  die  image  */ 

union  imagedptr  imgdata:  /’  ptrs  to  data  for  this  type  of  image  */ 

J: 

lypedef  struct  image  NPSIMAGE;  /*  define  an  NPSIMAGE  type  */ 
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C.  DATA  TYPE  HEADER  FILE:  MATCH_TYPES.H 

+  +  +  *  +  +  +  m  +  *  +  +  +  +  +  +  +  +  +  +  +  +  *  +  +  +  +  *  +  +  +  +  +  +  +  +  +  +  +  +  ****************** 

t*  file:  maich_iypes.h 

/* 

/*  This  File  holds  the  structure  definitions  for  image  edges  and 
/*  match  types  to  model  lines. 

/* 

/*  Types  defined:  EDGE 
/*  POINT 

/*  POSE 

/*  MATCHTYPE 

/*  IMG_L1NE 

y****************************************************************^*******^ 

typedef  struct  edge_region_type 

{ 

int  active;  /*  boolean  if  past  region  is  appended  to  a  present  region  */ 
long  first_pixel;  /*  first  and  last  pixels  added  to  region  */ 
long  last_pixel; 

long  xmin;  /*  min  &  max  pixels  for  previous  row  */ 
long  xmax; 

double  avg_phi;  /*  lor  use  with  dynamic  averaging  of  gradient  orientation,  phi  */ 
double  sum_phi; 

/*  Least  Squares  Fit  momments:  */ 
long  mOO;  /*  Number  of  pixels  */ 

double  mlO:  /*  Sum  x  */ 

double  mOl;  /*  Sum  y  */ 

double  mil;/*  Sum  x*y  */ 

double  m20;  /*  Sum  x*x  */ 

double  m02;  /*  Sum  y*y  */ 

struct  edgc_rcgion_type  *next;  /*  ptr  to  the  next  EDGE  */ 

}  EDGE; 


struct  point_type 

double  x,y;  /*  x,y  coordinates  of  the  pixel  endpoints  */ 

); 

typedef  struct  point_type  POINT; 
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struct  pose_type 
float  x,  y,  theta; 

}; 

typedef  struct  pose_type  POSE; 


typedef  struct  match_type 

{ 

LINE  *iine;  /*  ptr  to  LINE  type  (from  Jim  Stein’s  “graphics. c”)  */ 

double  angle_view_diff;  /*  angular  difference  between  image  and  model  lines  in  the  image  */ 

float  coni';  /*  confidence  value  for  the  match  */ 

float  dist;  /*  distance  between  the  *match  LINE  and  1MG_LINE  */ 

float  scale;  /*  ratio  l\IG_LlNE->dmajor  /  MODEL_LINE->length  */ 

struct  match_typc  *ne\i; 

)  MATCHTYPE; 


typedef  struct  line_tvpe 

{ 

char  name[ 3): 

POINT  pi.  p2;  /*  the  2  endpoints  lor  die  line  */ 

struct  line_type  ’next:  /*  ptr  to  the  next  IMG_LINE  in  the  image  */ 

/*  Least  Squares  Fit  momments: . - . ’/ 

long  mOO;  /*  Number  of  pixels  ’/ 

double  ni  10;  r  Sum  \  */ 

double  mOl;  /*  Sum  >  */ 

double  mil;  /*  Sum  x*y  */ 

double  m20;  /*  Sum  x“.\  */ 

double  m()2;  /*  Sum  \  *y  */ 

double  phi;  /*  Calculated  normal  orientation  of  1MG_L1NE  */ 
double  dmajor;  /*  Length  of  major  axis  of  equivalent  ellipse  */ 
double  dininor;  /*  Length  of  minor  axis  of  equivallcnt  ellipse  */ 
double  rho;  r  Ratio  dminor/dmajor  */ 
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/*  Pattern  Matching  Information: . */ 

double  angle_to_image_center; 

MATCHTYPE  *matchlist;  /*  List  of  matches  to  LINE  types  */ 
MATCHTYPE  *pm;  /*  present  match  being  considered  */ 

}  IMG_LINE; 
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APPENDIX  B-  EDGE  EXTRACTION  ROUTINES 


A.  IMPLEMENTATION:  F1NDEDGE.C 


y*  ******** X*************************^*****:*** *********************: 

/ *  FILENAME:  lindedge.c 
/*  AUTHOR:  Kevin  Peterson 
/*  DATE:  05  December  1991 

/* 

/*DESCRIPTION:  An  image  gradient  program  incorporating  edge-finding. 

/*  Displays  edge-gradient  image  and  associated  lines. 

/* 


/*  This  application  is  designed  t'or  use  on  a  Siiicon- 
/*  Graphics  Iris  ‘r>  workstation  unliving  a  .sgi 
/*  or  similar  rgb  formatted  image. 

/*  RGB  values  are  oflypc  LONG  in  the  form  AABBGGRR  where: 
I*  AA  -  alpha  value,  0-255 
/*  BB  -  ulue  component,  0-255 
/*  GG  -  green  component,  0-255 
/*  RR  -  red  component.  0-255 

r 


I*  SiliconGraphics  graphics  library  functions  used  within 
/*  the  display  _bw_and_gradient_images()  routine: 

/*  qdcviccO.  w  inseti ),  c3f(),  move20.  draw2(), 

/*  swapoufferst ),  reshapeview portf),  winclosetj. 

/*  and  Irectw  ritei >. 

/* 

/*  NPS1MAGE  function  routines  borrowed  courtesy  of  M.Zydu: 
/*  rcad_sgi_rgbimage(i,  get_empty_rgba_npsimage(), 

/*  get_empi\_rc’b_npsiniaiiei  e  and  rebalone  to  bwlonuu. 
r 


»*  *y 


/*  Leas  Squares  Fit  method  for  line-finding  from 
I*  "Sviiiar  Data  Interpretaion  for  Autonomous  Mobile  Robots" 

/*  by  V.Kanayama.  T. Noguchi.  2c  B. Hartman,  1990. 

/*  *****»»*,.*»»«  *»  »'■****»»*  »»•««(*»»»»»»»»>  :«»*******•»*■**/ 

#include  <gl.h>  /’  SiliconGraphics  (r)  graphic  library  */ 

#indude  <gl/image.h>  /*  SGI  image  structure  library  */ 

^include  <device.h>  /*  Machine-dependent  device  library  for  keys  ana  a.,-..  .e-buttons  */ 
^include  <stdiv>  h>  /“  C'  standard  i  b  library  */ 

# include  <nm.:..!i>  ’  C  math  library  for  aian2()  */ 


^include  ‘‘image_types.h"  r  Type  definitions  for  NPSIMAGE,  etc.  */ 
^include  "edge_types.h’- ./’  Type  definitions  for  EDGE,  LINE,  etc  */ 


#  include  "npsimagesupport.lv'  /*  Some  NPSIMAGE  functions  */ 

^include  “edgesupport.h"  /*  EDGE  and  IMG_L1NE  building  functions  */ 
^include  "display>up|x>rt.h"  /*  Graphics  display  functions  */ 

^define  THRESHOLD  5W(X)(Xk0/*  for  gradient  magnitude  threshold  lest  */ 
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main(argc,  argv) 
intargc; 
char  *argv[]; 

NPSIMAGE  *imgl,  /*  input  file_name.rgb  col'  mage  */ 

*img2,  f*  black&white  image  */ 

*img3;  /*  gradient  image  */ 

I*  pointers  to  RGBA  longs  (“bitsptr”s)  of  respective  NPSIMAGEs  */ 
long  *ptrl,  *ptr2,  *ptr3; 

double  dx,  dy,  Th  =  THRESHOLD*THRESHOLD; 
register  int  i  =  0,  /*  counter  for  pixels  in  input  image  */ 
z  -  0;  /*  counter  for  pixels  in  gradient  image  */ 

EDGE  *reg; 


if(argc  !=  2)  fatal(”usage:  findedge  filenameVi”); 

/*  Read  in  input  rgb  image  */ 
imgl  =  read_sgi_rgbimage(argv[l]); 
if(imgl  ==(NPSIMAGE  *)  NULL) 

{ 

latalC'Filc  %  s  is  a  NULL  image.\n”,imgl->name); 

) 

Xdim  =  imgl->xsize;  /*  else  set  global  Xdim  and  ptrl  ’/ 
ptrl  =  imgl->imgdata.biisptr; 

printf(“findcdge:>  %s  xsizc=  %d  ysize=  %d  pixels=  %cT\n”, 

imgl  ->name,imgl  ->xsize,imgl->ysize,  (imgl  ->xsize*imgl  ->ysize)); 


/*  Declare  new  NPSIMAGEs  */ 

if((imgl ->lypc  ==  RGB  A  WITH  ALPHA)  II  (imgl->type  ==  RGBA)) 

( 

img2  =  geLcmpty^gba^psimagefXdimjmgl^ysize.imgl^nameg 
img3  =  i!ct_<-'nipi)_rgba_npsimage((Xdim-2),imgl->ysize-2, ’’findedge”); 

} 

else 

{ 

latal(” Unknown  or  c-mapped  image  type:  %d.\n”,imgl->iypc); 

) 

ptr2  =  img2->imgdaia.bitsptr; 
ptr3  =  img3->imgdata.bitspir; 

/*  The  scan  of  an  RGB  image  is  from  the  bottom  row  ->  up, 
traversing  the  rows  left  to  right.  */ 

/*  In  order  for  the  Sobel  operator  to  be  calculated  for  a  specific  pixel, 
all  eight  surronding  pixels  must  have  an  absolute  (black  &  white) 
light  intensity  calculated.  Function  rgbalong_to_bwIong()  performs 
this  task.  */ 

/*  Due  to  the  nature  of  the  Sobel  operator,  the  pixels  in  the  top  and 
bottom  rows  as  well  as  pixels  in  the  leftmost  and  rightmost  columns 
of  the  input  image  will  not  be  calculated.  In  order  to  start  cal¬ 
culating  the  Sobel  operators,  the  first  2  rows  of  the  input  image 
must  be  converted  to  black  &  white  light  intensity  values.  */ 
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/*  Calculate  bw  values  for  first  2  rows  of  input  image.  */ 
for(i=0;  i<(2*Xdim)+2;  -r+i) 

( 

rgbalong_to_bv,  longtptrl  [i),&ptr2[i]); 

) 


for(i  =  Xdim  +  1;  1  <  (Xdhn*(imgl->ysize-l))-l;  ++i) 

( 

/*  Convert  color(imgl)  to  b/w(img2)  for  pixel  on  next  row  up  and  one 
pixel  over  to  the  right  so  that  all  eight  neighbors  of  pixel  i 
have  black  &  white  light  intensities.  */ 

rgbalong_to_bwlong(ptrl[i+Xdim+l],&ptr2[i+Xdim+l]); 

/*  Ensure  pixel  i  is  not  in  leftmost  ro  rightmost  column  */ 
if((i%Xdim  !=  0)  &&  (i£Xdim  !=  Xdim-1)) 

( 

r  Calculate  d.\,dy  via  Sobel  operator  for  pixel  i.  */ 

dx  =  f-ptr2[i+Xdim-l]  +  ptr2[i+Xdim+l] 

-(2  *  ptr2[i-l ]>+  (2  *  ptr2[i+l]) 

-ptr2[i-Xdim- 1 1  +  ptr2[i-Xdim+l]); 

dy  =  t,  ptr2( i+Xdim- 1 1  +  (2*ptr2[i+XdimJ)  +  ptr2[i+Xdim+ 1  ] 
-ptr2[i-Xdim-l  1  -  (2*ptr2[i-Xdim])  -  ptr2(i-Xdim+l]); 


il\(dx*d.\ )+(d>  *d\ )  >  Th) 

1 

pixel_membcrship(z,atan2(dy,dxj); 

set_pixel_black(&ptr3|zli: 


e  1  sc 
l 

Net_piwl_xhite<A;ptr3I/]); 


->-+/: Iik  remeni  the  pixel  counter  /.  lor  the  gradient  image.  */ 


r  It  pixel  i  is  in  the  leftmost  column,  do  chcck_acuvc_cdgesO.  */ 
else  iliiG  Xdim  ==  0> 

I 

chec  k_act  i  \  e_cdges(j; 

1 

/ 

}  /*  endfor  i  ’/ 

r  Check  remaining  EDGEs  lor  lines:  */ 
reg  =  Past_edge_list_head: 
whilefreu  1=  NULL) 

{ 

line_test(reg); 
reg  =  reg->ne\t: 


/*  Write  the  lines  list  to  file  "lines. text”.  *7 
write_alljines(argv[  1  ],img3->xsize,img3->ysizcj; 
printf(“Number  lines  found  in  %  s  =  7cdW’,  argv[l],Linecount); 

/*  Display  die  black&white  and  gradient  images  on  the  screen.  */ 
display_bw_and_gradicm_images(img2,img3,Linc_lisl_head); 

printlC’findedge  'v s...done.\n",argvll  ]); 
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B.  IMPLEMENTATION:  FASTEDGE.C 


/********************************************»»**********»************y 

/*  FILENAME:  fasiedge.c 
/*  AUTHOR:  Kevin  Peterson 
/•DATE:  06  January  1992 

/•  DESCRIPTION:  An  image  gradient  program  incorporating  edge-finding. 

/•  Displays  edge-gradient  image  and  associated  lines. 

/* 

/*  This  application  is  designed  for  use  on  a  Silicon- 
/•  Graphics  Iris  (r)  workstation  utilizing  a  .sgi 
/•  or  similar  rgb  formatted  image. 

/•  RGB  values  are  of  type  LONG  in  the  form  AABBGGRR  where: 

/•  AA  -  alpha  value,  0-255 
/•  BB  -  blue  component,  0-255 
/•  GG  -  green  component,  0-255 
/*  RR  -  red  component,  0-255 

/*  SiliconGraphics  graphics  library  functions  used  within  the  display_bw_and_gradient_imagesO 
/*  routine:  qdeviceO,  winsetf),  c3f(),  move20,  draw2(),  swapbuffersO,  reshapeviewportO,  wincloseO, 
/*  and  lrectwriie(). 

/*  NPSIMAGE  function  routines  borrowed  courtesy  of  M.Zyda:  read_sgi_rgbimageO, 

/*  get_empty_rgba_npsimage(),  get_empty_rgb_npsimage(3,  and  rgbalong_to_bwlong(). 

/*  Least  Squares  Fit  method  for  fine-finding  from  “Sonar  Data  Interpretaion  for  Autonomous  Mobile 
/•  Robots”  by  Y.Kanayama,  T.Noguchi,  &  B. Hartman,  1990. 
/**********************************************************************/ 

♦♦include  <gl.h>  /*  SiliconGraphics  (r)  graphic  library  */ 

#include  <gl/image.h>  /*  SGI  image  structure  library  */ 

♦♦include  <device.h>  /•  Machine-dependent  device  library  for  keys  and  mouse-buttons  */ 

♦♦include  <stdio.h>  f*  C  standard  i/o  library  •/ 

^include  <math.h>  /*  C  math  library  for  atan20  */ 

#include  “image_types.h”  /•  Type  definitions  for  NPSIMAGE,  etc.  */ 

#include  “edge_types.h”  /*  Type  definitions  for  EDGE,  LINE,  etc  •/ 

#include  “npsimagesuppori.h”  /*  Some  NPSIMAGE  functions  */ 

♦(include  “edgesupport.h”  /'*  EDGE  and  IMG _LINE  building  functions  */ 

#include  “displaysupport.h”  /•  Graphics  display  functions  */ 

mainfargc,  argv) 
int  argc; 
char  *argv[); 

{ 

NPSIMAGE  *img;  /•  input  file_name.rgb  color  image  •/ 
long  *ptr;  /*  pointer  to  bitsptr  of  NPSIMAGE  */ 
long  grmask  =  OxOOOOffOO; 
double  dx.  dy.  Th  =  THRESHOLD*THRESHOLD; 
register  int  i  =  0,  f*  counter  for  pixels  in  input  image  */ 

z  =  0;  /*  counter  for  pixels  in  gradient  image  */ 

EDGE  *reg; 

iffarge  !=  2)  fatal(“usagc:  fastedge  filename\n”); 

/*  Read  in  input  rgb  image  */ 
img  =  read_sgi_rgbimage(argv[l]); 

if(img  =(NPSIMAGE  *)  NULL)  fatal(“File  %s  is  a  NULL  image.\n”,img->name); 
printf(“fastedge  >  %s  xsize=  %d  ysize=  %d  pixels=  %dSn”, 
img->name,img->xsize,img->ysize,  (img->xsize*img->ysize)); 

Line_list_head  =  fastlines(img); 

display_lineJmage(img,Line_list_head); 
printf(“fastedgc  %s...donc.^n”,argv[l]); 

) 
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C.  IMPLEMENTATION:  YERTEDGE.C 


/*  FILENAME:  vertedge.c 
/*  AUTHOR:  Kevin  Peierson 
/*  DATE:  30  January  1992 
/* 


/*  DESCRIPTION:  Same  a  fastedge.c  but  only  returns  vertical  edges  from  image. 

»  *  *  **  *  »,»***/ 


#include  <gl.h>  /*  SiliconGraphics  (r)  graphic  library  */ 

#include  <gl/image.h>  /*  SGI  image  structure  library  */ 

#include  <device.h>  /*  Machine-dependent  device  library  */ 

/*  lor  keys  and  mouse-buttons  */ 

#include  <stdio.h>  /*  C  standard  i/o  library  */ 

#indudc  <math.h>  /*  C  math  library  foratan2()  */ 

^include  “imagc_types.h”  r  Type  definitions  for  NPS1MAGE,  etc.  */ 
#include  “cdge_t>  pes.h"  /*  Type  definitions  for  EDGE,  LINE,  etc  */ 

#include  “npsimagesuppori.h”  /*  Some  NPSIMAGE  functions  */ 

#include  “edgesupport.h”  /*  EDGE  and  IMG_L1NE  building  functions  */ 
#include  “vertsupporLh”  i*  Vertical  EDGE  and  IMG_LINE  supplement  */ 
#include  "displaysupport.h"  /*  Graphics  display  functions  */ 


mainfarge,  urgv; 
int  arge; 
char  *argv[  |; 

{ 

NPSIMAGE  *img:  r  input  lilejiame.rgb  color  image  ’/ 

iffarge  !=  2t  fatah "usage,  \ertedge  filcnume\n’\>: 

I*  Read  in  input  rgb  image  ’/ 
img  =  rcad_sgi_rgbimagetargv|  1  j); 

iffimg  —(NPSIMAGE  *>  NULL;  fatal(“Fi!c  fls  is  a  NULL  image.Nn”,img->name); 
printf(“vertedge>  !<s  \si/.e=  ysi/.e=  9<d  pixcl.s=  %d\n”, 
img->name,img->\si/e,img->ysi/.e,  (img->xsi/e*img->>  si/ein 

Line_lisl_heaJ  =  certlme.sumg  i: 

displayl  me._  i  magei  img,Linc_lisl_head); 

pnmip'vertedge  U  '...done.m",argv|  1 1): 
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D.  FILE:  NPS1MAGESI  PPORT.H 


/*  FILENAME:  npsimagesupport.h 
/*  AUTHOR:  Kevin  Peterson 
/*  DATE:  29  January  1992 
/* 

/*DESCR1PT10N:  Collection  of  basic  npsimage  functions. 

/*  NPSIMAGE  *get_empty_rgb_npsimage  (long  xsize,  long  ysize,  char  name[]) 
/*  NPSIMAGE  *gei_empty_rgba_npsimage  (long  xsize,  long  ysize,  char  name[]) 
/*  NPSIMAGE  *read_sgi_rgbimage  (char  filename!]) 

/*  void  writc_sgi_rgbimage  (char  filename!],  NPSIMAGE  *img) 

/* 


/* . . 

/*  NPSIMAGE 
/* 


. .  */ 

,'get_ernpty_rgb_npsimage  (long  xsize,  long  ysize,  char  name!]) 


/*  The  following  function  reads  in  an  SGI  RGB  image  as  an  NPSIMAGE. 
/*  -  eoutesy  of  M.  Zvda.  Naval  Postgraduate  School 
/* */ 


NPSIMAGE  *gci_empi>_rgb_npsimagc(xsizc, ysize, name) 
long  xsize,  ysi/.e;  /*  the  .size  the  image  should  be  in  pixels  */ 
char  name!  j:  /*  name  to  attach  to  the  image  */ 


NPSIMAGE  *img:  /*■  ptr  to  an  NPSIMAGE  */ 


r  allocate  an  NPSIMAGE  header  */ 

img  =  (NPSIMAGE  Mmalloctsi/.eofiNPSIMAGE)): 

I*  set  the  type  of  NPSIMAGE  to  RGBA  ’/ 
img->type  =  RGBA: 

/*  record  the  «  idths  and  height  *7 
img->xsi/.c  =  xsi/e: 
img->ysize  =  >m/c: 


/’  allocate  space  for  the  name  of  the  image  */ 
img->name  =  iclur  *  nnalloc(.strlen(name)+l); 

I"  copy  the  name  into  the  firing  allocated  * / 
sircpy(inig->name,namei: 

r  allocate  the  memory  lor  the  biunap  of  the  image  */ 
img->imgdaia.bitsptr  = 

(long  *)malloc(sizeol(longi  ’  nng->xsize  *  img->ysize): 

/*  return  a  ptr  to  this  empty  image  */ 
return(img): 
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r . */ 

/*  NPS1MAGE  *get_empi>_rgba_npsimage  (long  xsize,  long  ysize.  char  name[]) 
/* 

/*  The  following  function  reads  in  an  SGI  RGB  image  as  an  NPSIMAGE. 

/*  -  coulcsy  of  M.  Zyda,  Naval  Postgraduate  School 
/* .  . --*/ 

NPSIMAGE  *gci_einpiy_rgbu_npsintage(xsize,ysize,name) 

long  xsi/c,  ysizc;  /*  the  si/.e  the  image  should  be  in  pixels  */ 

char  namcl):  I*  name  to  attach  to  the  image  */ 


NPSIMAGE  *ung:  /*  ptr  to  an  NPSIMAGE  */ 


/*  allocate  an  NPSIMAGE  header  */ 

img  =  (NPSIMAGE  *  imalloc(sizeof(NPSIMAGE)): 

/*  set  the  type  of  NPSIMAGE  to  RGBA  */ 
img->type  =  RG  BANS'  1TH  ALPHA; 

/*  record  the  widths  and  height  */ 
img->xsi/.c  =  xsize; 
img->ysi/.e  =  \  si/e: 

r  allocate  space  for  the  name  of  the  image  */ 
img->name  =  tchar  *jinalloe(sirlen(nameH-l); 

/*  copy  the  name  into  the  string  allocated  ’ ' 
strep)  (img->name,namei: 

I*  allocate  the  memory  lor  the  bitmap  of  the  image  ’/ 

img->imgdaLi.bits|Ur  =  dong  *  jmalloc(si/.eof(Iong)  *  img->xsi/e  *  img->ysixe): 

/*  return  a  pir  to  this  empty  image  */ 
returni  img  i: 
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r . v 

/*  NPSIMAGE  ,,rcad_sgi_rgbimage  (char  filcnamc(]) 

/* 

/*  The  following  function  reads  in  an  SGI  RGB  image  as  an  NPSIMAGE. 
/*  -  coutcsy  of  M.  Zyda,  Naval  Postgraduate  School 
/* .  */ 


NPSIMAGE  *read_sgi_rgbimage(filename) 
char  filenamcl  ]:  /’  input  filename  */ 


register  IMAGE  ’image; 


NPSIMAGE  *img;  /*  pir  to  an  NPSIMAGE  structure  */ 

register  int  x,y;  /*  temp  indices  for  each  line  of 
data  from  the  sgi  image.  V 

long  ’pir;  /"  temp  pointer  for  each  word  of  the 
NPSIMAGE  long  image.  */ 

short  rbuf[4096|.  /*  temp  arrays  to  hold  scratch  info  for  */ 
gbuf[40%l,  /*  proeesMiie  an  sei  image  */ 
bbuf[40%). 
abuf[40%|; 


/*  open  an  .sgi  image  */ 

iff  iimaue=u>pem  filename.  "r"o  ==  NULL  ) 

l 

Iprintb stderr.  read  sei_rebimaee:  can't  open  input  tile  Ls\n'  .iilename); 
returmi NPSIMAGE  :  ’NULL 


if(image->/si/e<o » 

{ 

IprintKstderr.  read_sei_rebimaee:  this  is  not  an  RGB  imaee  lilcNn  i: 
returno NPSIMAGE''  .NLLLk 


/*  here  v.e  should  allocate  an  NPSIMAGE  */ 
if(image->/si/e  ==  3; 

( 

/*  just  allocate  an  rgb  image  */ 

img  =  gel  _empi\_rgb_npsirnage(image->xsi/.e,unage->ysi/.e,  filename); 


I*  gel  an  image  uli  alpha  *7 

ime  =  eei_emptv_rebn_npMmaize(image->xsizc,imai:c->vM/e,  filename); 

) 


/*  get  a  pointer  to  the  NPSIMAGE  longs  */ 
pir  =  img->imgdala.bilsptr; 
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/*  for  each  row  of  the  image  ...  */ 
for(y=0;  y  <  img->ysize;  y=y+l) 

( 

/*  read  a  row'  ot  reds  *7 
getrow(iniage,rbuf,y,0); 

P  read  a  row  of  greens  */ 
geirow(image,gbuf,y,l); 

P  read  a  row  of  blues  */ 
gcLrow(image,bbul,y  ,2  j; 

P  if  w'e  have  an  image  with  alpha,  gel  ii  */ 
if(img->iype  ==  RGB  A  WITH  ALPHA; 

[ 

geiro\\(  image. abul,y,3j; 


P  w'C  must  now  step  across  the  row  and  set  each  long  integer 
of  the  NPS IMAGE  formal  by  combining  the  info  from  the  sgi 
rows. 

V 

for(.\=0:  \  <  tmg-a>\M/e:  x=.x-rl; 


P  compete  the  RGBa  long  to  plug  in  and  plug  n  in  *7 
ill  img- >  type  =  =  RGB  AW  1THALPHA) 

J 

"pir  =  rhul ; x '  -r  t^bull x J  «  8)  +  (bbul'lxj  «  16)  +  < abul'[x ]  «  24) 


oNc 

i 

i 

.  ’  RGBA  image  with  alpha  forced  toOxlf  *7 
1  pu  =  rbul,  \ ;  tgbul'lx!  <<  8,'  +  (bbul'x)  «  16)  +  tOxff  «  24): 


P  step  the  j  to  the  next  long  *>• 

pir++: 

)  P  end  lor '  ■ 


P  we  have  ,->ei  all  t:,e  owes  ol  the  image. 

Now  letmi.  ill,  pu  to  the  NPS1MAGE  structure. 
"7 

reiurm  line  i: 
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/* . . */ 

/*  void  write_sgi_rgbiinage  (char  filenamc[],  NPS1MAGE  *img) 

/* 

/*  The  following  function  wiles  an  NPSIMAGE  to  a  file. 

/*  -  coutesy  of  Nl.  Zyda,  Naval  Postgraduate  School 
/* .  V 

void  wriie_sgi_rgbimage(filename,  img) 
char  filename!  j;  /'  output  filename  */ 

PSIMAGE  *ung;  /'  ptr  to  an  NPSIMAGE  structure  */ 


register  IMAGE  'image;  /'  a  ptr  to  an  sgi  image  structure  */ 

register  ini  x.y;  /'  temp  indices  for  each  line  of  data  from  the  sgi  image.  */ 

long  *pir;  /*  temp  pointer  for  each  word  of  the  NPSIMAGE  long  image.  */ 

long  dimen;  /’  dimension  ol  this  image  */ 

short  rbuf[4096|,  /*  temp  arrays  to  hold  scratch  info  lor  */ 
gbufl4096).  /*  proeesMiie  an  siti  image  */ 
bbuf[40%], 
abu '1409b): 


/’  set  the  dimension  and  /.size  of  this  imaee  '/ 
if(img->tvpe  ==  RGBAW1TH  ALPHA) 

( 

dimen  =  4; 

I 

else 

( 

/*  RGBA  image  o-  tliuiui  alpha  '/ 
dimen  =  V 


r  open  an  sgi  rgb  image  lor  writing  */ 

image=iopenUilcnamc,"tt  ",RLE(,1 ),  dimen,  img->xsize,img->\si/.e,dimen); 

I’  get  a  pointer  to  the  NPSIMAGE  longs  '/ 
ptr  =  img->imgdnia.biispir: 

/*  for  each  row  of  the  image  ...  '/ 
for(y=();  \  <  img->\si/e:  y=wli 


r  we  must  now  step  across  the  row  and  decode  each  long  integer 
of  the  NPSIMAGE  format  into  the  16  bit  shorts  sgi  requires  */ 

for(.\=():  \  <  imi:->.\si/e:  x=x+l ) 

( 

r  get  die  colors  Irom  the  longs  */ 
rbuf[x)  =  'ptr  k  OxOOOOOOff; 
gbufi x |  =  ('ptr  k  OxOOOOlfOO)  »  8; 
bbufjxj  =  ('ptr  k  0x00110000)  »  16; 
abuljxl  =  t'ptr  UxffUOOOOO)  »  24; 
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/*  step  the  ptr  u>  the  next  long  */ 
ptr+i-; 


/*  write  a  row  of  reds  */ 
putrow(image,rbuf,y ,().!: 

/*  w'rite  a  row  of  greens  *■/ 
putrow  (image  .gbul.y.l): 

/*  write  a  row  of  blues  *7 
putrow(image,bbuf,y,2): 

/*  write  a  row  of  alphas,  if  any  */ 
if(img->lvpe  ==  RGBAWITHALPHA) 

f 

putrow  umaee.abul  ,3  >: 

I 


r  we  must  elose  the  output  sgi  image  file  */ 
idose(imagei: 
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E.  FILE:  EDGESUPPOKT.il 


/**************************:t.***************¥**x************^******¥***/ 

/*  FILENAME:  edgesupport.h 
I*  AUTHOR:  Kevin  Peterson 
/*  DATE:  29  January  1992 

/* 

/^DESCRIPTION:  Collection  of  edge  finding  functions. 

/* 

/*  void  fatal(char  message) 

/*  int  gradient_angles_close(double  r, double  s) 

/*  int  close_to_negative_pi(double  phi) 

I*  int  horizontal(EDGE  *r) 

/*  EDGE  *crcate_edge(Iong  z, double  phi) 

/*  void  add_pixel_to_edge  (long  z,  double  phi,  EDGE *r) 

/*  EDGE  *combine_edges(EDGE  *rl,  EDGE  *r2) 

/*  IMG_LINE  *create_line  (EDGE  *r,  double  M20,  double  Mil,  double  M02, 

/*  double  Dmajor,  double  Dmmor,  double  Rho) 

/*  void  line_test  (EDGE  *r) 

/*  void  check_active_edges  () 

/*  void  rgbalong_tu_bwlong  (long  rgbalong,  long  *bwlong) 

/*  void  pixcLmembcrship  '  long  z,  double  phi) 

/*  void  set_pixel_white  (long  ’‘rgbalong) 

/*  void  set_pixel_blavk  (  long  ’rgbalong) 

/*  void  wrne_all_lines  (long  x,  long  y) 

/*  IMG_LINE  *  lastlines  (N  PS  IMAGE  *img) 

/*  Gradient  Magnitude  Threshold  -  for  fastlinesomt;  */ 
define  THRESHOLD  31)000.0 

/*  Gradient  Angular  Onenunion  */ 

^define  MA\_bEt-TA_PHl  0.5  /’  maximum  difference  tin  radians)  */ 

/*  Constants  lor  luiuii'in  void  !inc_tesi  (EDGE  *ri  */ 

Adeline  MIN_P1XELS_PER_L1NE  60/*  minimum  pixels  allowed  fora  IMG_LINE  */ 
Adeline  MIN_DMAJOR  20.0/*  minimum  major  axis  length  allowed  */ 

Adeline  MA\_RHO  0. 1  maximum  ratio  (Rho=Dminor'Dmajor)  */ 

Adeline  PI  3.141 59205 


/*  ---  Global  variables . */ 

long  Xdim:  /*  width  of  input  image  (nr  pixels)  */ 
long  Ydim;  /*  w  idth  of  input  image  (nr  pixels  */ 

int  Linecount  =  0;  /*  counter  for  number  of  IMG  LINEs  made  ’/ 


r  Pointers  to:  Present  row  EDGE  list.  Past  row  EDGE  list,  IMG_LI\E  list*/ 

EDGE  *Prcsent_edge_list_hcad  =  NULL 
*Presem_edi:eJisi_iail  =  NULL, 

*Past_edgc_fist_head  =  NULL; 

IMG.LINE  *  Line'll  ist_heatl  =  NULL; 
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/*  . 

/*  void  fatal  (char  menage) 

/* 

/*  Prints  error  message  and  exits  out  ol'  the  program. 

/* . V 

fatal(message) 
char  ‘message: 

( 

fprintl(stderr,  “Fatal  ERROR:  "); 

perror(message); 

exit(-l);  /*  exit  bv  failure  */ 

} 


/* . . */ 

/*  int  gradient_angles_close  (double  r,  double  s) 

I* 

/*  Returns  1  if  gradient  angle  orientations  of  EDGEs  r  and  s 
/*  are  within  MAX_DELTA_PHI. 

Returns  0  others',  ise. 

I* . *7 

int  gradicni_angles_closeir.s) 
double  r.s: 

( 

if((r  <  MAX  DELTA_PH1  -  PI)  &&  (s  >  0.0))  r  =  Pi  +  PI  +  r; 
else  ilUs  <  MAX_DELTA_PHI  -  PI}  (r  >  0.0) >  s  =  PI  +  PI  +  s: 

return: labsir  -  ■'  MAX_D1-.;.TA_PHI): 


r . */ 

/*  int  close_U'_neeaiivc  p:  'double  phi) 

/* 

/*  Returns  1  if  orientaiion  phi  is  within  MAX_DELTA_PHI  to  -PI. 
/*  Returns  U  otherwise 

r . . *V 

int close_lo_neg.:  \e..puphi i 
double  phi: 

1 

return  H  -  phi  e  MAX.  DLL  I  A_PHlj: 


r . */ 

/*  int  hori/onLil  i  EDGL  * e t 

r 

t*  Returns  1  il  orientaiion  of  EDGE  e  (r->avg_phi)  is  within 
I*  MAX_DEL1  A_PHI  -1  to  PI  2  or  -PI/2  (the  normal  orientations  for 
/*  a  horizontal  line). 

/*  Returns  U  other*  ise 

/* . . . */ 

int  horizonuilie  i 
EDGE  *e: 

I 

double  inaxphi  =  MAX_DELTA_PHI  /4.0; 

return((fabs(e 's  ivg_phit  >  (0.5* PI )-  .axphi)  && 

(fab.sie->avg_phii  <  (0.5*PI)+maxphi)): 

) 
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r . - . - . 

I*  EDGE  *create_edge  (long  z,  double  phi) 

r 

I*  Returns  pointer  to  an  newly  instantiated  EDGE  with  variable 
/*  based  upon  the  input  pixel  z  and  orientation  phi  of  pixel  z. 

/* . . -*/ 

EDGE  *create_cdge(z,  phi) 
long  z;  /*  zth  pixel  in  image  */ 
double  phi;  f*  gradient  orientation  of  pixel  z  */ 

{ 

EDGE  *r; 

long  x  =  z%(Xdim-2),/*  (x,v)  coordinates  of  pixel  z  in  2D  imattc  */ 
y  =  z/(Xdim-2); 

/*  allocate  memory  for  EDGE  r  */ 

if((r  =  (EDGE  *)malloc(Mz.eof(EDGE)))  ==  NULL)  faul(“create_edge:  malloc\n”);  ) 

/*  else  initialize  fields  of  EDGE  r  */ 

r->active  =  0; 

r->first_pixel  =  z; 

r->last_pi\el  =  z: 

r->xmtn  =  x; 

r->xmax  =  x; 

r->avg_phi  =  phi; 

r->sum_plti  =  phi; 

r->m00  =  1 ; 

r->ml()  =  x; 

r->m()l  =  y; 

r->m  1 1  =  x*y: 

r->m20  =  x“x: 

r->m()2  =  y*y: 

r->next  =  NULL: 

il'(Present_edge_hst_head  ==  NULLi 
Present_edge_lisi_liead  =  r: 
il(Present_edge_li.st_uiil  !=  NULL) 

Present_edge_list_taii->next  =  r: 

Present_edge_list_lail  =  r; 

returm  rn 
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/* . */ 

/*  void  add  pixel_to_edge  (long  z.  double  phi,  EDGE  *r) 

I* 

/*  Updates  EDGE  r  with  new  pixel  z  and  orientation  phi  of  pixel  z. 

/* . */ 

add_pixel_to_edge(z,  phi,  r) 
long  z;  /*  zth  pixel  */ 

double  phi;  /*  gradient  orientation  of  pixel  z  */ 

EDGE  *r;  /*  EDGE  to  add  pixel  z  to  */ 

{ 

long  x  =  z%(Xdim-2),  /*  (x,y)  coordinates  ol  pixel  z  in  2D  image  */ 
y  =  z/(Xdim-2): 


if((r->avg_phi  >  0.0)  elose_to_negative_pi(phi)) 

{ 

phi  =  PI  +  phi  +  PI: 

} 

else  il((phi  >  0.0)  AiAi  close_to_negative_pi(r->avg_phi)) 

{ 

r->avg_phi  =  PI  +  r->a'.g_phi  +  PI; 
r->sum_phi  =  r->a\  it_phi  ¥  r->m00; 

} 


/*  update  die  new  xmax  for  this  row  and  last_pixel  added  to  this  EDGE  */ 
r->xmax  =  x; 
r->last_pixel  =  /:. 

/*  update  the  least  squares  lit  moments  */ 

++r->m00; 
r->ml0  +=  x; 
r->m01  +=  y: 
r->ml  1  +=  x*\: 
r->m20  +=  x*x; 
r->m()2  -!■=  y*\; 

r  recalculate  the  average  gradient  orientation  for  the  EDGE  */ 

r->sum_phi  +=  phi; 

r->avg_phi  =  r->suni_plu  /  r->m()0; 
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/* . 

/*  EDGE  *eombine_edges  (EDGE  *rl,  EDGE  *r2) 

/* 

/*  Returns  pointer  to  EDGE  rl  alter  appending  all  information 
/*  of  EDGE  r2  with  rl. 

/* . . V 

EDGE  *combine_edi’es(rl  ,r2) 

EDGE  *rl,  *r2; 

{ 

/*  Special  case  where  rl->first_pixel  is  changed:  */ 

if(!((r2->first_pixel  >  rl->first_pixel)  II  (horizontal(rl)  &.&.  (rl->xmin  <  r2->xmin)))) 

l 

rl ->1  ir.st_pi.xcl  =  r2->lirst_pixel: 

} 


/*  Modify  lasl_pixel  */ 
if(r2->lasl_pixel  >  rl->lasi_pixel) 

! 

rl->last_pixel  =  r2->lasi_p»xcl; 
rl->xmax  =  r2->\ma\: 

} 

else  if(hori/.onialtr2)  A:  A:  ir2->xmax  >  rl->xmax)) 

1 

rl->last_pi\el  =  r2->last_pixel: 


/*  Special  case:  if  gradient  angles  on  opposite  sides  of  +/-  pi  */ 
if((rl->avc_phi  >  0.0}  A: A:  dose_to  ncgative_pi(r2->avg  phi)) 

( 

r2->avg_phi  =  PI  +  r2->a\g_phi  +  PI: 
r2->sum_phi  =  r2->a\e_phi  *  r2->m00: 

) 

else  il((r2->a\  e_pln  >  U.n>  A. A:  close_lo  neuative  pi(rl->avg_phi}) 

{ 

rl->avg_phi  =  PI  -r  rl->avg_phi  +  PI; 
rl->sum_phi  =  rl->a\g_phi  *  rl->m00: 


/*  combine  least  squares  In  moments  */ 

rl->m00  +=  r2->inOO: 

rl->ml0  +=  r2->ml(); 

rl->m01  +=  r2->m()l: 

rl->ml  1  +=  r2->m  1 1 . 

rl->m20  +=  r2->m20; 

rl->m()2  +=  r2->m02: 

/*  combine  and  recalculate  average  gradient  orientation  lor  the  EDGE  */ 
rl->sum_phi  +=  r2->sum_phi; 
rl->avg_phi  =  rl->.sum_phi  /  rl->m(X); 

returni  r  1 1;  /’  return  EDGE  rl  with  combined  information  of  rl  A:  r2  */ 
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r . •/ 

I*  IMG_LINE  *create_line  (EDGE  *r,  double  M20,  double  Ml  1,  double  M02, 
/*  double  Dinajor,  double  Dminor,  double  Rho) 

/* 

/*  Returns  pointer  to  newly  instantiated  1MG_LINE  with  variables  set 
/*  according  to  moments  described  by  EDGE  r,  secondary  moments 
/*  M20,  Ml  1 ,  and  M02,  axis  lengths  Dmajor,  Dminor,  and  ratio  of 
/*  axis  lengths  Rho. 

/* .  . V 

IMG  LINE  *create_Iine(r,M20,M  1 1  ,M02,Dmajor,Dminor,Rho) 

EDGE  *r; 

double  M2(),M  1 1  ,M02, Dmajor, Dminor, Rho; 


IMG_LINE  *1; 

/*  r->first_pixel  mapped  onto  the  IMG_LINE  will  be  endpoint  pi 
r->last_pixcl  mapped  onto  the  1MG_LINE  will  be  endpoint  p2  */ 


long  xl  =  r->first_pixel%(Xdim-2), 
yl  =  r->first_pixel/(Xdim-2), 
x2  =  r->lasl_pixel9<(Xdim-2), 
y2  =  r->last_pixel/(Xdim-2); 


/*  Calculate  the  normal  orientation  of  the  1MG_LL\'E  by  atan2^  function.  */ 

double  Pin  =  utan2t-2>  M 1 1  ,M02-M2())/2.0, 

/*  Delta  1  and  delta2  are  the  offsets  used  to  calculate  the  endpoints 
for  the  1MG_LINE  segment  based  upon  values  xl.yl  and  \2,y2.  */ 

delta  1  =  ir->mlO/T->niOU  -  tdoublejxl  )*fcos(Phi)  + 

(r->in()l/r->m()0  -  (double))  l)*fsin(Phi), 
deltaZ  =  (r->ml()/r->m()U  -  tdoublc)x2)*fcostPhi)  + 

(r->m()l/r->mO()  - 1 double )y 2)* fsin(Phi); 


/*  Phi  =  aUtn2t-2*M  1 1  ,MO2-M20)/2.0)  always  returns  positive  result  to  Phi. 

Therefore,  negative_phi  =  tr->a\g_phi  <  0.0)  is  necessary  .  ’/ 

ini  ncgativc_phi  =  ^r->avg_phi  <  0.0); 

r  Allocate  memory  for  1MG_L1NE  1.  *1 

if((l  =  (IMG.LINE  *)malloe(si/.eof(IMG_LINE)})  ==  NULL)  fatal(“creatc_line:  mallocNn”); 


/*  Calculate  x.v  coordinates  for  endpoints  pi  and  p2.  */ 

l->pl.x  =  (doublc)xl  +  delta  l*fcos(Phi); 
l->pl.y  =  (double))  1  +  delta  l*fsin(Phi); 
l->p2.x  =  (doublcjx2  +  delta2*fcos(Phi); 
l->p2.y  =  (double))  2  +  delui2*fsin(Phi); 
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/*  Copy  least  squares  lit  moments.  */ 


l->m00  =  r->m(J0; 
l->mlO  =  r->mlO; 
l->m01  =  r->m01; 
l->mll  =  r->ml  1; 
l->m20  =  r->m2(); 
l->m02  =  r->m()2; 


/*  Phi  is  positive,  but  -pi  <  r->avg_phi  <  pi.  */ 

if(negati  ve_phi)  l->pln  =  -Phi; 
else  l->phi  =  Phi; 


/*  Update  rest  of  IMG_LINE  v  alues.  V 


l->next  =  NULL; 

l->dmajor  =  Dmajor; 
l->dminor  =  Dminor; 
l->rho  =  Klto; 

l->matchlisi  =  Nl  LL; 
l->pm  =  NULL; 


++Linecouiu;  /*  Increment  global  v  ariable,  Linecount.  */ 
strcpy(l->name,'' 

sprintf(l->name,  "‘ul".  Linecount); 
rcturn(l):  /*  Return  1MCI_LINE  I.  */ 

I 
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/* . 

I*  void  line_lest  (EDGE  *rj 

/* 

/*  Determines  if  EDGE  r  meets  three  requirements  to  be  a  IMG_L1NE: 

/*  (1)  The  number  of  pixels  in  EDGE  r  (r->mOO)  be  greater  than  MIN_P1XELS_PER_LINE. 
/*  (2)  The  ratio  (Rho)  of  the  length  of  major  and  minor  axes  of  the 
/*  EDGE  be  less  than  MAX_RHO. 

/*  (3)  The  length  of  the  major  axis  (Dmajor)  be  greater  than  MIN_DMAJOR,  the  minimum 
/*  1MG_LINE  length  allowed. 

/*  If  all  three  conditions  are  met,  a  new  IMG_L1NE  type  is  created  and 
/*  appended  to  the  Line  list  in  order  of  significance  (in  this  case  Dmajor). 

/* .  . */ 

line_test(r) 

EDGE  *r: 

( 

1MG_LINE  *1,  *insert_pt  =  Line_list_head; 

double  M20.M  1 1  ,M()2,Ma,Mb,Mmajor,Mminor,Dmajor,Dminor,Rho; 

/*  First  test  --  A  IMG_L1NE  must  have  a  required  minimun  number  of  pixels.  */ 
if(r->m()()  >  M1N_P1.XELS_PER_LINE) 
i 

/*  Calculate  secondary  moments  by  least  squares  fit.  */ 

M20  =  r->m20  -  (tr->m  10*r->ml0)/r-:>m00); 

Ml  1  =  r->ml  1  -  ((r->ml0,'r->m01)/r->m00); 

MU2  =  r->m()2  -  (tr->m0l*r->m01)/r->m00); 

/*  Calculate  major  and  minor  axis  lengths,  Dmajor  and  Dminor.  */ 

Ma  =  (M20+M02J/2.0; 

Mb  =  sqrt(  ((M02-M20)*(M02-M20)/4.0)  +  (Ml  I’M  1 1 )  ): 

Mmajor  =  Ma  -  Mb: 

Mminor  =  Ma  +  Mb; 

Dmajor  =  4.0,,sqrt(Mminor/r->m00): 

Dminor  =  4.0,:si|rii.Mmaioi7r->m()0); 

I*  Calculate  ratio  Rho.  ”, 

Rho  =  Dminor/Dmajor. 

/*  Second  &  Third  tests  --  Ratio  Rho  must  represent  a  line,  not  a  blob. 

--  IMG  LINE  must  be  at  least  a  certain  length.  */ 
if((Rho  <  MA\_RHO)  Cc&  (Dmajor  >  MlN_DMAJORj) 
i 

/*  The  EDGE  passed  the  three  requaments  to  be  a  line.  */ 

I  =  ereate_line(r,M20,M  1 1  ,M02, Dmajor, Dminor, Rho); 

/*  Add  new  1MG_L1NE  to  IMG_L1NE  list  in  order  by  IMG_LINE  length,  dmajor.  */ 
if(Line_list_head  ==  NULL)  Line_lisi_head  =  1: 
else  il\l->dmajor  >  Line_lisl_hcad->dmajor) 

( 

l->next  =  Line_hst_head: 

Line_list_head  =  1: 


) 


while((insert_pi->next  !=  NULL)  &&.  (l->dmajor<  inseri_pi->next->dmajor)) 
inscn_pt  =  insert_pt->nexi; 
l->next  =  inseri_pt->ncxl; 
insen_pt->ne.\i  =  1; 

) 

}  /*  end  i!  second  and  third  tests  */ 

)  r  end  if  first  test  */ 
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/* . */ 

/*  void  chcck_aciive_edges  () 

/* 

/*  This  funcuon,  performed  at  the  start  of  scanning  for  each  row, 

/*  cycles  through  the  EDGEs  in  the  Present_edge_list  to  see  if: 

/*  (1)  Adajaccnt  EDGEs  on  the  same  row  have  gradient  orientations 
/*  that  are  close  and  therefore  can  be  combined  into  one 
/*  EDGE. 

/*  (2)  EDGEs  from  the  Prescnt_edge_list  are  adjacent  to 
/*  EDGEs  from  the  past  row  ’s  Prcvious_edgc_list  and  their 
/*  gradient  orientations  are  close  so  that  they  may  be 
/*  combined  into  one  EDGE. 

/*  The  second  method  for  determining  w'hcn  to  combine_edgcs() 

/*  provides  the  means  lor  constructing  EDGEs  across  rows. 

I* 

/*  All  EDGEs  not  combined  with  a  EDGE  from  the  past  row  are 
/*  tested  by  linc_tcst()  which  examines  the  EDGE  for  satisfying 
I*  the  IMG  JJNE  requirements  and  appends  the  IMG_L1NE  to  the  Linejist. 

t* . */ 

check  active_edees() 

l 

EDGE  "pres  =  Present_edge_list_head,  /*  the  EDGEs  found  by  the  scan  of  this  row  */ 

*pasi  =  Pasi_edge_iisi_hcad,  /*  the  EDGEs  found  during  the  scan  of  the  previous  row  */ 
*temp:  /*  a  temporary  pointer  used  for  freeing  memory  */ 

ini  continue_loo|): !'  a  boolean  integer  */ 


I*  Loop  through  all  EDGEs  in  the  Present_edgc_list  (all  EDGEs  found 
during  the  scan  of  the  present  row.  */ 

whiletpres  1=  NL'LL  > 


r  Look  lorward  to  tile  next  EDGEs  on  the  present  row  to  see  if 
any  should  be  combined  with  diis  EDGE.  If  so,  coinbine_edgcsu 
and  freer  i  the  second  EDGE.  */ 

while((pres->ne\t  '=  NL’LL)  &.&.  (pres->xmax+l  ==  prcs->next->xmin)  && 
eradient_amtles  closc(pres->avg  phi,prcs->ncxt->ave  phi)) 

1 

|ires  =  combine_edges(pres,pres->ncxt); 
temp  =  pres->next: 
pres->nexl  =  pres->next->next; 
f  reel,  tempi: 


I*  Set  continue_loop  boolean  to  true  and  loop  through  the  EDGEs  of 
found  during  the  scan  of  the  previous  row  of  pixels.  */ 
continue  Joop  =  1: 

whileiipast  1=  NLLLi  CcJe  (pres->xmax  >  past->xmin-2)  (continuc_loop)) 


r  If  the  two  EDGEs  are  adjacent  and  their  gradient  orientations 
are  close,  then  the  information  of  the  past  EDGE  must  be  in¬ 
cluded  with  the  present  EDGE  and  the  EDGE  should  be  marked 
that  a  is  still  "active”.  */ 
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if((pres->xmin  <  past->xmax+2)  &.&  gradient_anglcs_closc(past->avg_phi,pres->avg_phi)) 

{ 

pres  =  combinc_edges(pres,past); 
past->activc  =  1 ; 


r  If  the  past  EDGE  was  not  appended  to  a  present  EDGE,  (i.c. 
not  “active")  then  no  more  pixels  may  be  added  to  it  and  it 
shall  be  tested  if  it  satisfies  the  conditions  for  an  1MG_LINE.  */ 

if((past->nc.\i  !=  NULL)  &.&  (pres  >xmax  >  past->ncxt->xmin-2)) 

i 

i t(!pasl->aeiive)  line_test(past); 
past  =  past->next: 


/*  If  the  past  EDGE  did  not  meet  any  of  the  above  two  conditions, 
then  exit  this  loop  and  continue  evaluation  with  the  next 
EDGE  of  the  present  row.  */ 

else  conimuc_kx>p  =  0; 


pres  =  pres->nc\t: 


I*  Continue  line_tesii  >  and  I'reet )  all  EDGEs  in  the  past  row .  */ 
whilefpasi  1=  NL  LLi 
{ 

temp  =  past; 

if('past->acti\et  line_iesii.pasi); 
past  =  past->ne.\i: 
freei  tempi: 


/*  Finally:  Fast  row  <-  Present  row ,  Present  row  <-  empt> .  */ 
Past_edge_list_head  =  Presem_edge_list_head; 
Presenl_edge..lisi_heaJ  =  NULL; 

Pre.seni_edgejisi_lail  =  NL  LL; 
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/* . •/ 

/*  void  rgbalong_to_bwlong  (long  rgbalong,  long  ’bwlong) 

/* 

/*  Converts  color  to  black/white  for  rgba  formatted  pixels. 

/*  The  weights  assigned  for  each  color  are  television  standards. 

/*  This  function  courtesy  of  M.  Zyda. 

/* . . . */ 

rgbalong_to_bwlong(rgbalong, bwlong) 
long  rgbalong;  /*  input  color  rgbalong  */ 
long  ’bwlong;  /*  output  b/w  rgbalong  */ 

{ 

unsigned  char  red,  green,  blue,  alpha; 
unsigned  bw; 

/*  Use  bit  masks  to  get  RGB  and  alpha  values  from  input  rgbalong  */ 

red  =  rgbalong  &  OxOOOOOOff; 

green  =  (rgbalong  &  0x00001(00 j  »  8; 

blue  =  (rgbalong  &.  0x001(0000)  »  16; 

alpha  =  (rgbalong  Ac  0\IT(X)0000)  »  24; 

/*  Calculate  the  blacks  white  intesity  using  NTSC  standard, 
intensity  =  0.299(red)  +  0.587(grcen)  +  0.1 14(bluc)  */ 
bw>  =  (0.299’red  M().587*green)+(0. 1 14*b!uc): 

/*  Save  the  black&white  intensity  in  bwlong.  */ 

’bwlong  =  tal|iha«24  i!(bv.  <<  I6)l(bw«8)lbw ; 


/* . ’/ 

/*  void  pixel  niember.'hip  done  /.,  double  phi) 

/* 

/’  For  a  given  row  while  scanning  the  black&w  hite  image,  (scan  of  row>  ....yyyyy....rrrrrrrrrz 
/*  y  =  pixels  included  in  previous  EDGE  of  same  row, 

/*  r  =  pixels  included  in  EDGE  r, 

/*  l  =  pixel  /  beinu  tested  for  inclusion  with  EDGE  r) 

/* 

/*  tw  o  requirements  must  be  satisfied  for  a  pixel  (z)  with  the  pixels  of  a  given  EDGE: 

/*  (1)  pixels  must  be  adjacent  (i.e.  the  lasi_pixel  of  the  EDGE  +  1  must  equal  pixel  z), 

/*  (2)  the  gradient  angle  orientations  of  the  last_pixe!  and  z  must  be  “close’’.  Otherwise,  pixel  z 
I*  can  not  be  included  w  ith  EDGE  rand  must  be  considered  the  firsi_pixcl  of  a  new  EDGE  for  this  row. 

/* . V 

pixcl_membership(/,  phi ) 
long  z;  r  the  zth  pixel  ul  the  gradient  image  */ 
double  phi: 

( 

EDGE  *r: 


if((Present_edge_list_tail  1=  NULL)  &.& 
(Present_edge_lisi_iail->Iasi_pixel+l  ==  z)  &.& 
gradient  angles_elose(Presenl_edgeJist_iail->avg_phi,phi)) 
{ 

add  pixel_lo_edeei/,phi,Present_cdge_list_tail); 

) 

else  /*  create  new  EDGE  and  add  to  end  of  Present  list  ’/ 

I 

r  =  create_edeei/.phi): 

) 
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/* . •/ 

/*  void  set_pixel_white  (long  ’rgbalong) 

/* 

/*  Sets  the  specified  long  integer  pointed  to  by  rgbalong  to  be 
/*  white  by  setting  all  RGB  bits  to  ff  (255  dec  ->  max  intensity). 

/* . - . V 

sei_pixcl_white(rgbalong) 
long  *rgbalong; 

( 

*rgbalong  =  Oxl  1 1 11111: 

) 


/* . . ----•/ 

/•  void  set  pixcl_blaek  (Iona  ’rgbalong) 

/• 

/*  Sets  the  specified  long  integer  pointed  to  by  rgbalong  to  be 
/*  black  bv  setlina  all  RGB  bits  to  00  (0  dec  ->  min  intensity). 

/* . . - . */  ' 

sct_pixel_black(rgbalong  > 
long  ’rgbalona: 

l 

•rgbalong  =  OxllOOUOOO: 


/* . •/ 

/•  void  wriie_all_lines  done  x,  lone  y) 

/* 

/•  Write  lo  output  file  "name. text”. 

/* . ’/ 

write_all_linesi\.y ) 

lone  x.v;  /•  the  x,\  dimensions  of  the  image  */ 

i 

1MG.L1NE  •!  =  Linejistjicad; 

FILE  •lines  file: 


lines_file  =  fopenC’lincs.iexi’Vw”); 

fprinif(lines_file.”*d  J  ‘i  d\n\n’\x,y); 

whilc(l!=MLL) 

( 

lprintl(lines_lile.  ‘^.2i  ‘<.2l\n9i.2l  %.2l\n9i.4l\n”, 
l->pl.x.l->pl.\  .l->p2.x,l->p2.y,l->phi): 

I  =  l->nexi: 


fdose(lines_file): 

prinifl"  lines  found  in  image  written  to:  ‘lines. texl’Nn”): 


% 


r  */ 

/*  IMG  LINE  *  lastl  inest,N  PS  IM  AGE) 

F 

/*  Returns  a  poinicr  10  the  head  of  a  linked  list  ol  vertical  IMGJJNEs  lound  in  an  NPS1MAGE. 

/* . */ 

1MG_LINE  *fasllines(img) 

NPSIMAGE  *img;  /*  input  filc_name.rgb  color  image  */ 


long  *ptr;  /*  pointer  to  bitsptr  of  NPSIMAGE  */ 
long  grmask  =  OxOOOOffOO; 
double  dx,  dy,  Th  =  THRESHOLD*THRESHOLD; 
register  int  i  =  0,  /*  counter  for  pixels  in  input  image  V 

z  =  ();  /*  counter  for  pixels  in  gradient  image  */ 
EDGE  *reg; 


Xdim  =  img->xsi/e; 

Ydim  =  img->ysiz.e; 
ptr  =  img->imgdaia. bitsptr: 

Linecount  =  0: 

/*  The  scan  of  an  RGB  image  is  from  the  bottom  row  ->  up,  traversing  the  rows  left  to  right.  */ 
l'or(i  =  Xdim  +  1:  i  <  tXdim*(img->y$ize-l))-l;  ++i) 


F  If  pixel  i  is  in  the  leftmost  column,  do  chcck_aciive_edges().  */ 
if(if/tXdim  ==  0>  check_active_cdges(); 

F  Ensure  pixel  i  is  not  in  leftmost  or  rightmost  column  */ 
else  i f ( i Xdim  !=  Xdim- 1 1 

( 

F  Calculate  dx.dv  via  Sobel  operator  for  pixel  i.  */ 

dx  =  (-(ptrji+Xdim- 1  LCgrmask)  +  tptrli+Xdim+lJ&grmaskj 
-i2  *  iptr|i- 1  !>kgrmask))+  (2  *  (pir[i+l]&grmasky) 

-i.ptr[ i- Xclim- 1  ,Ckgrmask)  +  (pu(i-Xdim+l  l&grmaskjK 

d\  =  (  (pirji+Xdini- 1  |<kgrmaskj  +  (2*(ptr[i+Xdimj&grmask)) 
+  tptr|i+Xdim+l)&grmask)  -  (ptr[i-Xdim-l]&grmask) 

- 1 2*< ptr[ i-Xdim jdC grmask))  -  tplr[i-Xdim+l]&grmask)): 

il'Udx“d.\  i-t-idv *d\  >  >  Th)  pixel_mcmbership(./.,aian2(dy,dx)); 

tr/.; 


1  F  endlor  i  */ 

F  Check  remaining  EDGEs  lor  lines:  */ 
reg  =  Past_edge_list_head: 
whiletree  !=  NULL) 

( 

line_iest(reg): 
reg  =  reg->ne\t: 


F  Writ,  the  lines  list  to  tile  "lines. text”.  */ 
writc_alljincs(img->xsize,img->ysize); 

printlf"  Number  lines  found  in  C s  =  %d\n”,img->name .Linecount); 

rcturn(Line_list_head>: 

} 
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F.  FILE:  DISPLAYS t'PFOKT.H 


^fr*************************************************************^*****)! ¥  j 

/*  FILENAME:  displaysupport.h 
/*  AUTHOR:  Kevin  Peterson 
/*  DATE:  31  January  1992 
/’DESCRIPTION:  Collection  of  display  functions. 

/* 

/*  void  displav_bw_and_gradicnt_images  (NPSIMAGE  *imgl,  NPS I  MAGE  *img2, 

I*  IMG_L1NE  *1) 

/*  void  draw_red  Jines  (1MG_LINE  *1) 

/*  void  display_loop  (NPS1MAGE  *imgl,  long  winidl, 

/*  NPSIMAGE  *img2,  long  winid2) 

//*  void  display _linc_iinage  (NPSIMAGE  *img,  1MG_L1NE  ’1) 

/*  void  display_linc_!oop  (NPSIMAGE  *img,  IMGJJNE  *1,  long  winid) 

I* 

/*  These  functions  make  calls  to  following  SiliconGraphics  routines: 

/*  prefsize(),  winopcnO,  winsct(),  winclose(),  RGBmodeQ, 

/*  singlcbuffcrO,  gconfigO,  qdevice(),  c3f(),  move2 (),  draw2(), 

/*  swapbuffersf),  lrectwriteO,  reshapeview  portQ,  and  c!car(). 

/*  **»»***»>:,,,,*>*>>:**»  *****»*»****»**  *****  **************  **/ 

I” . */ 

/*  void  display _bv\  and  ttradicni_images  (NPSIMAGE  *imgl,  NPSIMAGE  *img2, 
r  "  IMG_LINE  *1; 

/*  Displays  NPSIMAGE  imgl  and  img2  on  SiliconGraphics’  Iris  workstation. 

/*  Each  window  will  be  displaved  within  red  outline  w  hen  operator  depresses  left  mouse  button. 

/* . - . */ 

display_bw_and_gradicm_images(imgl,img2,l) 

NPSIMAGE  ’imgl,  /*  input  b/w  image  */ 

*ime2:  /’  gradient  image  */ 

IMG_LINE  *1: 

I 

long  winid  I ,  w  inid2:  >'  silicon  graphics  w  indow  id’s  */ 

prelsizcOrng  1  ->.\si/.e,img  1  ->>  size );  /*  preferred  size  for  w  indow  */ 
winidl  =  wmopen(imgl->name);  /*  open  the  window  */ 

RGBmodcO:  /*  set  RGBmode,  singlebuffer,  and  */ 
singlcbuffcrO:  /*  configure  the  window  */ 
gconfigO; 

prefsize(img2->.\siz.e,img2->)iiz.e);  r  preferred  size  for  w  indow  ’/ 
winid2  =  w  inopen(img2->nanie;;  /*  open  the  window  */ 

RGBmodcO;  /*  set  RGBmode,  singlcbuffer,  and  */ 
singlcbuffcrO:  /’  configure  the  w  indow  */ 
gconfigO; 

r  initialize_conirols  */ 

qdevice(REDRAW); 

qdcvice(LEFTMOUSE): 

qdevicc(MIDDLEMOUSE): 

qdcvice(RIGHTMOUSE): 

qdevicc(ESCKEY): 

display_loop(img  1  ,w  mid  1  ,img2,w inid2,I  y. 

frec(imgl->imgdata.bitsptrg  /*  delete  the  bitmap  for  the  image  */ 

free(imgl);  /*  delete  the  NPSIMAGE  structure  */ 

winclose(w inidl );  /*  clo.se  the  window  */ 

frcc(img2->imgdata.bitsptr): 

frec(img2); 

winclose(wimd2i; 
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r . 

/*  void  draw_red_lincs(.IMG_LINE  *1) 

/* 

/*  Draws  red  lines  over  an  image. 

/* .  -  . 

draw_rcd_lines(l) 

IMG_LINE  *1; 

( 

sialic  float  red  [  3 1  =  { 1 .0.0  0,0.0);  /*  rgb  red  */ 

c3f(red); 

while(l!=NULL; 

l 

move2(l->pl.x  l->p  1 . v ;; 
draw2(l->p2.x,i->p2.y): 

1  =  l->nexi; 

} 

swapbullersy; 


/* . 

/*  void  draw  whitc_lincsilMG_LlNE  *1) 

/* 

/*  Draw  s  while  lines  over  an  image. 

/* . . . 

draw_whiie_lines(l) 

1MG_L1NE  ’  1: 

( 

sialic  lloal  whiie|3|  =  j  1 .0. 1.0, 1.0}:  /*  rgb  while  */ 

c3l\w  hile): 
whilevl!=NL'LL  < 

I 

nujve2(l->pl.x,l->pl.\  >: 
draw 2il->p2.\.l->p2.\  >: 

1  =  l->nexi; 

) 

swapb'il  lersi  i: 


r 

/*  void  draw_blaek_line.M I.MG_L1NE  *1) 

/* 

r  Draws  black  lines  overall  image. 

/* . . . 

draw_blaek_linest  1 1 
1MG_L1NE  *1. 

I 

sialic  lloal  black! 3 i  =  d'.O.O.O.O.OJ;  /*  rgb  black  */ 

c3t(blaekK 
while(l!=NL  LL  < 

I 

move2(l->pl  ,x.l->pl .)  i: 
draw2(.i->p2.\,!->p2.\  i: 

1  =  l->nc\i: 


sw'apbullersi  >: 


r . •/ 

/*  void  display  Joop  (NPS1MAGE  *imgl,  long  winidl, 

/*  NPSIMAGE  *img2,  long  winid2) 

/* 

/*  Continuously  displays  the  white&w'hite  (imgl)  and  gradient  (img2) 
/*  images  within  their  respective  windows  (winidl  and  winid2). 

/*  -  Depress  left  mouse  button  when  cursor  arrow  is  in  utle 
/*  bar  to  see  lines  initially  over  the  image. 

/*  -  Depress  middle  mouse  button  when  cursor  is  in  image  to 
/*  see  lines  only. 

/*  -  Depress  middle  mouse  button  when  cursor  is  in  title  bar 
/*  “drag”  window  and  redisplay  image  with  lines. 

/*  -  Depress  right  or  left  mouse  buttons  when  cursor  is  i 
/*  either  image  to  kill  windows. 

/* . — . - . "7 

display_loop(imgl  .winid  1  ,img2,winid2,l) 

NPSIMAGE  *imgl,  /*  input  b/w  image  */ 

*img2;  /*  gradient  image  */ 
lone  winidl,  winid2:  /*  window  id’s  for  images  */ 

I.MGJJNE  *1: 


{ 

static  float  wlmei.lj  =  j  1.0.1.0.1.0J;  /*  rgb  w'hite  */ 
short  value:  /*  value  returned  from  the  event  queue  */ 

/*  display  the  images  once  */ 
winsetfwinidl  c 

lreciwriie(().(),imgl->\si/e  -  I  ,imgl->ysi/e  -  l.imgl^imgdata.bilsptr); 
winset(winid2t; 

lrectw  rite((),(),img2->.\si/e  -  I  ,img2->ysi/e  -  l,img2->imgdaui.biLsptrg 

lw  loop  until  a  mouse  button  is  pressed  */ 
whilefTKL'Ei 
{ 

sw  itchu|readuv  value  ■  • 

1 

case  REDRAW : 

winsei(.i.long)value>: 
reshape  view  ixvipn 

il\ value  ==  winidl  i  lreeiwnte(0,0,tmgl->.\size  -  l,imgl->ysi/e  •  1,  imgl->imgdata.bitsptr) 
il\v  alue  ==  vvimd2i  Ireetw  riiet0,0,img2->xsi/.e  -  I.img2->ysi/.e  -  1,  img2->imgdata.biLsptr) 
draw_red_lmesi  1  n 
break; 

case  LEFTMOUSE: 
case  MIDDLEMOl  SE: 
if(  value  ==  0)  1 
e  3 1\  w  hite  >: 
clean  k 

draw  red_line.stl»: 

1 

break: 

case  RIGHTMOUSE: 
ift. value  ==  t))  return: 
break: 

case  ESCKEY: 

e.xitiOn 

default: 

break: 

)  /*  end  switch  */ 

)  )*  end  w  hile  */ 
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I* . */ 

/*  void  display_linc_image  tNPSIMAGE  *img,  IMG_L1NE  *1; 

/* . . */ 

display_line_image(ime,l » 

NPSIMAGE  *img:  " 

IMG_L1NE  *1: 

! 

long  winid; 

prefsizc(img->xsize,img->ysize);  /*  preferred  size  for  window  */ 
winid  =  winopen(img->name);  /*  open  the  window  */ 
RGBniode();  /*  sei  RGBmode,  singlcbuffer,  and  */ 
singlebufferf):  /*  configure  the  window  */ 
gconfigO; 

/*  initialize_controls  */ 

qdevice(REDRAW); 

qdevicefLEFTMOL'SEj; 

qdevicc(MlDDLEMOL'SE); 

qdcvicc(RIGHTMOUSE>: 

qdevice(ESCKEY): 

display  _linc_ltx>p(img.l.w  mid,); 

frce(img->imgdaia.bilsptr);  /*  delete  the  biunap  for  the  image  */ 
frce(img);  /*  delete  the  NPSIMAGE  structure  */ 
windose(winid):  /’  close  the  window  */ 


r . */ 

/*  soid  displa\_lme_loop  iNPSIMAGE  *ima,  1MG_LINE  *1.  lone  w inidj 

r . — — . : . •/ 

display  _line_loopt  ime.l.w  mid  i 
NPSIMAGE  Nine: 

1MGJJNE  ”1: 
long  w  inid: 


static  Boat  w  Intel 3)  =  1 1.0, 1.0, 1.0  j:  /*  rgb  white  */ 
short  value:  r  \alue  returned  from  die  event  queue  */ 

/*  display  the  images  once  */ 
winset(winid): 

lreciwrite(.(),()amg->Nsi/.e  -  l.img->ysizc  -  l,img->imgdaia.biispir); 

/*  loop  until  a  mouse  button  is  pressed  */ 
whiletTRLE) 


sw  itchtqreadtA:  \  alue  >  • 


case  REDRAW: 

win.sei(tlong)value.i: 
reshape  v lewportt,): 

ill  value  ==  winid'  lrectwrite(0,0,img->xsize  -  l,img->ysi/.e  -  1,  img->imgdata.bilspir) 
break: 
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case  LEFTMOUSE: 

/*  draw  while  lines  over  inpul  image  */ 
if( value  ==  0)  draw  whiiejincs(l); 
break; 

case  M1DDLEMOUSE: 
if(valuc  ==  0) 

{ 

c  31  (while): 
clear(); 

draw_black_lincs(l): 

) 

break; 

case  R1GHTMOUSE: 
if(value  ==  0)  reiurn; 
break; 

case  ESCKEY; 
exii(O); 

deiauli: 

break; 

)  /*  end  switch  */ 

)  /*  end  while  "7 
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G.  FILE:  VERSl  PPORT.H 


/*  FILENAME:  vertsupport.h 
/*  AUTHOR:  Kevin  Peterson 
/*  DATE:  29  January  1992 

/* 


/’DESCRIPTION:  Supplementary  vertical  edge  finding  functions. 
/*  ini  venical(EDGE) 

/*  void  vcrtical_linc_iesi(EDGE) 

/*  void  chcck_aciive_vertical_edgcs() 

/*  IMG_LINE  *vertlines(NPS  IMAGE) 


Adeline  VERT_CONSTR AIN'T  0.0349  /*  radians  (=  2.0  degrees)  */ 


r . - . */ 

/*  ini  verticalfEDGEi 
/* 

/*  Returns  1  if  orientation  of  EDGE  r  (r->avg_phi)  is  within 
/*  MAX_DELTA_PHl/4  of  0,  PI,  or  -PI  (the  normal  orientations  for 
/*  a  vertical  line). 

/*  Returns  0  otherwise. 

/* . */ 

ini  veriicaliri 
EDGE  *r: 

( 

return  Rlahstr->a\e_phi)  <  VERT_C0NSTRAINT)  II 
(fahsi.r->a\g_phir>  PI  -  \  ERT_C0NSTRA1NT )): 


105 


r . *i 

I*  void  vcrlioal_line_iesl(EDGE) 

/* 

/*  Same  as  line_tesi(r),  bui  also  ensures  that  EDGE  r  is  vertical. 

I* . "7 

vertical_line_test(r) 

EDGE  *r; 

( 

1MG_LINE  ’1,  *inseri_pi  =  Line_list_hcad; 

double  M20,M  1  l,M02,Ma,Mb,Mmajor,Mminor,Dmajor,Dminor,Rho; 

I*  First  test  -  A  1.\1G_LINE  must  have  a  required  minimun  number  of  pixels.  */ 
if((r->mf:0  >  M1N_P1XELS_PER_LINE)  &&  (veriical(r))) 

{ 

/*  Calculate  secondary  moments  by  least  squares  lit.  */ 

M20  =  r->m20  -  ((r->mlO*r-:>mlO)/r->mOO); 

Ml  1  =  r->ml  1  -  ((r->ml0*r->m01)/r->m00); 

M02  =  r->m()2  -  ur-> mO  1  * r->mO  1  )/r->  mOO ) ; 

/*  Calculate  major  and  minor  axis  lengths,  Dmajor  and  Dminor.  */ 

Ma  =  (M20+M02J/2.0: 

Mb  =  sqrt(  aM02-M20r(M02-M20)/4.0)  + (Ml  1*M1 1)  j: 

M major  =  Ma  -  Mb: 

Mminor  =  Ma  +  Mb: 

Dmajor  =  4.0*,sqru.Mminor/r->m00); 

Dminor  =  4.0*sqri(Mmajor/r->m(X)); 

/*  Calculate  ratio  Rho.  ’/ 

Rho  =  Dminor/Dmajor; 

/*  Second  &  Third  tests  -  Ratio  Rho  must  represent  a  line,  not  a  blob. 

--  IMG_LINE  must  be  at  least  a  certain  length.  */ 
il'((Rho<  MA\_RHOi  vVcV  (Dmajor  >  MIN  DMAJOR)) 
t 

r  The  EDGE  passed  the  lour  requirments  to  be  a  vertical  line.  */ 

1  =  create  Jmetr.M2(),M  1 1  ,M02,Dmajor,Dminor,Rho): 

/*  Add  new  IMG_L1NE  to  1MG_L1NE  list  in  order  b> 

1MG_L1NE  length,  dmajor.  */ 
if(LineJisi_head  ==  NULL)  Line_list_head  =  1: 
else  ilil->dmajor  >  Line  list_hcad->dmajor) 

( 

l->next  =  Line_list_head: 

Line_list_head  =  1: 

) 

else 

I 

whilei.(insert_pt->ne\i  !=  NULL)  ik& 

il->dmajor  <  msert_pi->next->dmajor)) 

I 

insert_pt  =  msen_pt->next: 

1 

l->ne.\t  =  in>ert_pt->nexi: 
i»sert_pi->ne\l  =  1: 


)  r  end  il  second  and  third  tests  */ 
)  r  end  if  first  test  “/ 
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/* . */ 

/*  void  check  aclive_vertical_edgcs() 

/* 

/*  Same  as  check_aciive_cdgcs(),  bui  for  vertical  EDGEs  only. 

/* . . */ 

check_aciive_veriical_edgcs() 

( 

EDGE  *pres  =  Present_edgc  JislJiead, 

/*  the  EDGEs  found  by  the  scan  of  this  row  ♦/ 

♦past  =  Past_edgc  _lisi_head, 

/♦  the  EDGEs  found  during  the  scan  of  the  previous  row  ♦/ 

♦temp;  /♦  a  temporary  pointer  used  for  freeing  memory  */ 

int  continue  Joop;  /*  a  boolean  integer  */ 

/♦  Loop  through  all  EDGEs  in  the  Present_edge_list  (all  EDGEs 
found  during  the  scan  of  the  present  row.  ♦/ 

whilefpres  !=  NULLi 

( 

/*  Look  lorward  to  the  next  EDGEs  on  the  present  row  to  sec  if 
any  should  be  combined  with  this  EDGE.  If  so,  combinc_edgcs() 
and  freef.)  the  second  EDGE.  */ 


while((pres->next  !=  NULL)  &&  (pres->xmax+l  ==  pres->next->xmin)  && 
eradient_an«leN_clo^o(pres->avg_phi,pres->ne\l->avg_phi)) 

! 

pres  =  combine_edges(pres,pres->next); 
temp  =  pres->next: 
prcs->next  =  pres->ncxt->ncxi: 
lree(iempi: 


/♦  Set  continue_loop  boolean  to  true  and  loop  through  the  EDGEs  of 
found  during  the  ^an  of  the  previous  row  of  pixels.  */ 

continue  Joop  =  1 : 

while!. (past  !=  NULL)  (,pres->xmux  >  pasi->xmin-2)  &.&.  (coniinue_loop)) 


I*  If  the  tw  o  EDGEs  are  adjacent  and  their  gradient  orientations 
are  close,  then  the  information  of  the  past  EDGE  must  be  in¬ 
cluded  w  ith  the  present  EDGE  and  the  EDGE  should  be  marked 
that  n  is  still  "active".  ’/ 

if((pres->xmin  <  past->xmax+2)  && 
eradient_angles_close(past->avg_phi,pres->avg  phi)) 

I 

pres  =  eombine_edgcs(prcs,past); 
past->active  =  1: 


r  If  the  past  EDGE  was  not  appended  to  a  present  EDGE,  (i.e. 
not  "active”)  then  no  more  pixels  may  be  added  to  it  and  it 
shall  be  tested  if  it  satisfies  the  conditions  for  an  1MG_L1NE.  */ 

if((pasi->nexi  !=  NULL)  &&  (pres->xma.x  >  past->ncxt->xmin-2)) 

{ 

il(!past->aciive)  veriical_linc_tesl(pasi); 
past  =  pasi->next; 
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/*  If  the  past  EDGE  did  not  meet  any  of  the  above  two  conditions, 
then  exit  this  loop  and  continue  evaluation  with  the  next 
EDGE  of  the  present  row,  */ 

else  coniinucjoop  =  0; 


pres  =  pres->next: 


/*  Continue  vertical_line_iest()  and  free()  all  EDGEs  in  the  past  row.  */ 
whilefpast  1=  NULL) 

{ 

temp  =  past; 

if(!past->aaivc)  veriical_line_test(pasi); 

past  =  past->nexi: 

frcc(tcmp): 


/*  Finally:  Past  row  <-  Present  row ,  Present  row  <-  empty.  */ 
Past_edgc_list_head  =  Present_edge_list_hcad; 
Prcscnt_cdge_list_head  =  NULL: 

Present_edee_lisi_tail  =  NULL: 
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/* . v 

/*  1MG_L1NE  ’vcnlincstNPS  IMAGE) 

/* 

/*  Returns  a  pointer  to  the  head  of  a  linked  list  ol  vertical 
/*  IMG.LINEs  found  in  an  N'PSIMAGE. 

/* . */ 

1MG_LINE  *vertlines(img) 

NPS1MAGE  ’ling;  /’  input  file_name.rgb  color  image  */ 

{ 

long  *ptr;  /*  pointer  to  bitsptr  ol  NPS1M  AGE  */ 
long  grmask  =  OxOOOOffOO; 
double  dx,  dy,  Th  =  THRESHOLD’THRESHOLD; 
register  ini  i  =  0,  /*  counter  for  pixels  in  input  image  */ 
z  =  0;  /*  counter  lor  pixels  m  uradieni  image  */ 

EDGE  *reg; 


Xdim  =  img->xsize; 

Ydim  =  img->ysize; 
ptr  =  img->imgdata.biuptr; 
Linecount  =  (l; 


/*  The  scan  of  an  RGB  image  is  from  the  bottom  row  ->  up, 
traversing  the  rows  left  to  right.  */ 

lorfi  =  Xdim  +  I:  i  <  i Xdim*(img->ysize-l))-l;  ++i) 

{ 

/’  It  pixel  i  is  m  the  lelunosl  column,  do  check_active_edgesi  ).  */ 
if(i9iXdim  ==  Oi  chcck_aciive_veriical_cdgcs(); 

/*  Ensure  pixel  i  is  not  in  leftmost  or  rightmost  column  */ 
else  ilfi'i  Xdim  !=  Xdim- 1 ) 

I 

r  Calculate  dx.dy  \  ut  Sobel  operator  tor  pixel  i.  ’/ 

dx  =  (-tpir[i-rXdmi- 1  J&grmask)  +  (,pir|i+Xdim+lJ<Xgrmaski 
-i2  *  i.pu-|i-ljA;gnnask;>+  (2  *  (ptrfi+lj&grmask)) 

-i ptr li-Xdim- 1  Uv grmask)  -»■  fpir[i-Xdim+l]&grmask)); 

dy  =  ( (ptr|i+Xdim- 1  J&grmask)  +  (2*(ptr[i+XdimJ&grmask)> 
t  (ptrii+Xdim+l  j&grmask)  -  (ptrli-Xdim-l]&grmask) 

-  i.2*tpul,i-Xdmiii:grniask))  -  fptr[i-Xdim+l)&grmask)); 

il'Udx’dx  i+idx *d\ )  >  Thi  pixel_mcmbcrship(/.,auin2(ily,dx)); 
++/: 

) 

|  /’  endtor  i  ’/ 

/*  Check  remaining  EDGEs  lor  lines:  */ 
reg  =  Past_edge_lisi_head: 
whilefreg  !=  NULL) 
i 

veriical_line_iestireg  n 
reu  =  ree->ncxt; 

I 

/’  Write  die  lines  list  to  tile  ‘'lines.text”.  */ 
wriie_al!Jincs(jmg->xsi/.e,img->ysize); 

priiufy  ivumbci  lines  found  in  %s  =  ?rd\n’',img->name, Linecount); 
return(Line_lisi_head): 
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APPENDIX  C  -  LINE  MATCHING  AND  POSE  DETERMINATION  ROUTINES 


A.  IMPLEMENTATION:  V  ERTMATCH.C 

/*  FILENAME:  vcrtmaich.c 
/*  AUTHOR:  Kevin  Peterson 
/*  DATE:  09  March  1992 

/**************«**9^***%******************«*%******9| i*******************/ 

#include  <gl.h>  /*  SilieonGraphics  (r)  graphic  library  */ 

#include  <gl/image.h>  /*  SGI  image  structure  library  *1 

#include  <dcvice.h>  /*  Machine-dependent  device  library  for  keys  and  mouse-buttons  */ 
#include  <stdio.h>  /*  C  standard  i/o  library  */ 

#include  <math.h>  /*  C  math  library  for  atan2()  */ 

^include  “2d+.h”  /’Header  files  for  3D  model  environment  by  J.  Stein  */ 

#includc  “5th.h” 

#includc  “graphics. h” 

^include  "visibility. h“ 

#include  “image_types.h”  /’  Type  definitions  for  NPSIMAGE,  etc.  ’/ 

#include  "malch_types.li '  r  Type  definitions  for  EDGE,  IMG_LINE..  */ 

^include  "npsimagesupport.h"  /*  Some  NPSIMAGE  functions  */ 

#includc  “edgesuppon.h"  /*  EDGE  and  IMG _L1NE  building  functions  */ 

^include  "vertsupporl.h”  /’  Vertical  EDGE  and  1MG_L1NE  supplement  ’/ 

#indude  “maichsuppori.h"  /’  LINE  and  IMG_LINE  matching  routines  */ 

^include  "nutchdisplay  support. h“  /’  Graphics  display  functions  */ 


main(argc,  arg\  i 

ml  arge: 
char  *ar g\ ; ;: 
i 

WORLD  ’FifthFloor: 

NPSIMAGE  *ung; 

IMG  LINE  ’ImugeLine.sHead  =  NULL,  ’imageline; 

LINE.HEAD  ’ModelLmesHeadI  =  NULL,  *ModclLinesHead2  =  NULL; 

POSE  *in_posc,  ’corr_pose: 

float  xO,  y().  thelaO.  u_|xisit.  center_u_axis; 

if  (urge  !=  2 1  latah,  “usage:  \ertmalch  filcnamcNn”); 

/’  Pose  values  lor  data  images:  - . */ 

xO  =  60.0;  >0  =  366.0;  thetaO  =  250.0;  /*  060366250.pic  */ 

/*  xO  =  60.0;  >0  =  366.0;  tlietaO  =  253.0;  /*  060366253.pic  ’/ 

/’  xO  =  60.0;  vO  =  366.0:  thetaO  =  251.0;  /’  060366251. pic  »/ 

/*  xO  =  59.0;  \0  =  366.0;  thetaO  =  250.0;  /»  059366249. pic  ’/ 

/*  xO  =  94.0;  y()  =  381.0;  thetaO  =  245.0;  /*  094381 245.pic  */ 

/*  xO  =  68.0;  yO  =  372.0:  thetaO  =  249.0;  /»  068372249.pic  ’/ 

/’  xO  =  48.0;  yO  =  277.0.  thetaO  =  000.0;  /’  048277000.pic  */ 

/*  xO  =  48.0;  y()  =  277.0;  thetaO  =  002.0;  /’  048277002. pic  */ 

/*  xO  =  48.0;  >0  =  277.0;  thetaO  =  356.0;  /*  048277356.pic  »/ 

/»  xO  =  43.0;  yO  =  277.0:  thetaO  =  356.0;  /*  043277356.pic  */ 

/*  xO  =  43.0;  >0  =  267.0;  thetaO  =  000.0;  /*  043267000. pic  */ 

/»  x()  =  48.0;  \0  =  267.0;  thetaO  =  000.0;  /*  048267000.pic  */ 

/*  xO  =  48.0;  \0  =  2721):  thetaO  =  000.0;  /*  048272000.pic  */ 
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/*  Read  input  rgb  image  and  extract  vertical  edges.  */ 
img  =  read_sgi_rgbimagc(argv[  1  ]); 

if(img  ==(NPS1MAGE  NULL)  fatal(“File  %s  is  a  NULL  imagc.Sn”,img->name); 
printfC‘vcrtmaich>  9<  s  xsize=  %d  ysize«  %d  pixels-  %dSn”, 

img->natne,img->xsizc,img->ysize,  (img->xsizc*img->ysizc)); 

ImagcLinesHcad  =  vertlincs(img); 
imageline  =  ImagcLinesHcad; 
ccntcr_u_axis  =  (noat)(img->xsize/2); 
whilc(imagclinc  !=  NULL) 

{ 

u_posit  =  (imageline->pl.x  +  image!ine->p2.x;/2.0; 
imagclinc->am;le_io_image_center  =  atan2((cemer_u_axis  -  u_posit), 
F0CAL_LENGTH_1N_P1XELS); 
imageline  =  imagcline->next; 


/*  Declare  tlie  estimated  pose  ol  the  camera.  */ 
il'((in_pose  =  (POSE  *)malloc(sizeof(POSE)))  ==  NULL) 
falalC'crcaimg  POSE  in_posc:  malloc\n”); 
in_pose->x  =  .\0: 
in_posc->y  =  yU; 

in_pose->theta  =  normalize(lhetaO  *  (PI/180.0)); 

primf('*SnDR  (input)  pose:  x  =  9{.2f,  y  =  c/< .  2f.  theta  =  %.2f(5J.2f  radsj\n\n”, 
i  n_posc->\. m_posc->y, thetaO. in_pose->theta); 


/*  Initialize  world  database  lor  filth  floor  of  Spanagle  Hall,  determine 
2D  view  of  on\  ironment,  and  label  the  vertical  model  features.  */ 

FiflhFloor  =  makc_world(): 

ModelLineslleadl  =  uci_\icw(in_pose->.\.  in_pose->y,  CAMERA_HEIGHT,  thetaO, 

FiflhFloor.FOC  AL_LENGTH  K 

printf("Nr  V  ertical  Model  Lines  =  U dW.ModelLinesHead  1  ->VERT_L1NES); 
label_model_!inesi  ModcILmesllead  I  ->\  LINE_L1ST,  in_pose): 


/*  Call  to  updaie.poseyi  */ 

corr_pose  =  updatc_posci  lniagcLincsHead,ModelLinesHeadl->YLlNE_LIST,in_posc); 

thetaO  =  corr_pose->tlteui  *  180.0  /  PI; 
if(lhcia0  <  U.()  ■  thetaO  +=  S00.0; 

prinif('\nCorrecled  pose:  x=  U .21,  y=  %.2f,  thcut=  %.2f(9c.4f  rads)\n”, 
corr_pose->x.corr_posc->y,thctaO,corr_pose->iheia); 

r  Get  2D  view  of  world  Irom  corrected  pose.  */ 

ModelLinesHcad2  =  eel  view (corr_posc->x,  corr_pose->y,  CAMERA_HEIGHT,  thetaO, 

FiflhFloor,  FOCAL_LENGTH); 

/*  Displas  image,  2D  view  Irom  input  pose,  and  2D  view  from  corrected  pose.  */ 
display  jnaichjmaga img.  ImagcLinesHcad,  ModclLinesHcadl .  Mode!LincsHcad2); 


free_lines(ModelLinesHead  I  >: 
frcc_lines(M(xlelLmesHead2): 

print!)"  \ertmaich  U  s.  ,doiie.\n",argv(l)); 
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B.  FILE:  M ATCHS U PPORT.H 


/*  FILENAME:  matchsupport.h 
/*  AUTHOR:  Kevin  Peterson 
/*  DATE:  09  March  1992 

l+#1t  +  *  +  +  3tt  +  +  ***  +  +  +  *  +  itt*3li*3ri*.****  +  **********  +  *3ll************+***  +  *****  +  *****l 

#define  FOCAL_LENGTH  1 .40  /*  cm  */ 

#define  FOCAL_LENGTH_IN_PIXELS  1205.4524/*  number  of  pixels  */ 

#dcfinc  VERT_P1CTURE_B0RDER  8.0/*  number  of  pixels  */ 

#define  FIELD_HALF_ANGLE  0.2556  /*  radians  =  14.645  degrees  */ 

#dcfine  CAMERA_H EIGHT  40.0  /*  inches  */ 

Adeline  MlN_CHORD_LENGTH  10.0/*  min  dist  between  2  vertical 

model  features  (in  inches) 
used  in  dciermine_posiiion()*/ 

#define  DELTA, LENGTH_RATIO  0.05  /*  Endpoint  dclta/IMG_LINE  length 

used  in  deiermine_vcrt_maich()*/ 

Adeline  1MG_L1NE_.M1N_ANGLE  0.0070  /*  radians  =  0.20  degrees  */ 

Adefine  V  ERIFY_EQ_A\GLE_EPSlLON  0.0035  /*  radians  =  0.20  degrees  */ 

/’ . */ 

/*  double  normal  l/.euiuble  alpha) 

/* 

/*  Returns  an  ancle  between  pi  and  -pi. 

/* . . */ 

double  normali/.ei alpha) 
double  alpha: 
i 

whiletalpha  >  PH  alpha  -=  (2.0*P1); 
whiletalpha  <  -PI i  alpha  +=  (2.0* PI ): 
return  alpha: 


/* . */ 

/* 

Returns  I  it  ancle  a  1  is  "lei l_of  angle  a2. 

r . . */ 

V 

int  lcfl_ol(a  I ,  t»2) 

double  a  I ,  a2; 

I 

return  inormali/eial  -  a2)  >  0.0); 


r . •/ 

/* 

Returns  angle  for  use  with  environment  model  coordinates. 
Environment  model  uses  ancles  measured  from  y-axis. 

/* . - . - . */ 

*/ 

double  map_angle(alpha) 
double  alpha: 

i 

return  normah/etalpha  -  (PI/2.0));  /*  alpha_prime  */ 
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/* .  */ 

/*  void  label_modelJincs(LINE  *m,  POSE  *p) 

/* 

I*  Routine  to  determine  lengths,  orientations,  angles  from  image  center,  and 
/*  labels  for  all  model  LINEs  visible  in  image. 

/* . */ 

label_modcl_lincs(m,  p> 

LINE  *m; 

POSE  *p;  r  estimated  camera  POSE  */ 

1 


unsigned  char  first_char  =  '  ',  second_char  =  ‘A’; 


/*  Note:  Vertical  edge  extraction  produces  IMG_LlNEs  with  pi  .y  always 
less  than  p2.y.  Similarly  fur  vertical  model  LINEs,  Y1  is  always 
less  than  Y2. 

*/ 

prinilCVn.Model  Lincv - Mi”): 

whilefm  !=  NULL > 


m->!ength  =  m->Y2  -  m->Yl; 

m->esi_|)ose_orient  =  map_angIe(atan2(m->MODEL_Y  -  p->y, 
m->\10DEL_X  -  p->x)); 

m->esi_angle_to_image_cenier  =  m->cst_pose_orient  -  p->theta; 

ilim->length  >  15.0)/*  temporary  -  don't  do  very  small  lines  */ 

i 

-»uvp\  im->name,” 

Nprimfi.m->name,  "9< o%c*\  Iirsl_char.  second_char): 

ih  wond_char  ==  Z  i 

ii\first_char  ==  ‘  ')  first _char  =  A’; 
else  first_char  =  first_char  +  1: 
second_char  =  *A’; 

I 

I 

eUe  second_char  =  second_char  +  1; 

pmnir  %s:  (%.!!,%, If)  length=  %.2f,  psi=  % .21,  psi-lhctaO=  %.4f\n”, 
m->namc,  m->MODEL_X,  m->\10DEL_Y,  m->length, 
m->esi_posc_oricnt,  m->est_angle_to_imagc_ccntcr); 


i 


spiiiiilim->name,  “  "r. 


m  =  m->\EXT: 
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r . */ 

/*  void  detcrminc_vcn_malch(LLNE  ’m,  1MG_L1NE  ’i) 

/* 

/*  Function  to  determine  confidence  value  for  a  possible  match 
/*  based  upon  translation,  rotation,  scaling,  and  overlapping 
/*  of  endpoints  between  a  LINE  from  the  model  list  and  an 
/*  IMG_LINE  from  the  image  list. 

/* .  » . . . V 

determ  ine_verijnaic  h(  m  ,i ) 

LINE  *m;  /’  ptr  to  LINE  from  model  list  */ 
1MG_L1NE  ’i;  /*  ptr  to  IN1G_L1NE  from  image  list  ’/ 
l 

ini  endpoint_inclusion; 
double  delui; 

MATCHTYPE  ’match,  ’place: 


/*  Endpoint_Inclusion  of  image  endpoints:  for  vertical  image  lines,  i->pl  is 
always  lower  than  i->p2.  ’ ' 

delta  =  tm->Y  I  -  i->pl.\ )  -r  ti->p2.y  -  m->Y2); 

if((dclu  <=  0.0)  II  ((della  >  0.0)  &&  (dclta/i->dmajor  <  DELTA_LENGTH_RATIO))) 

i 

;*  llicn  create  MATCHTYPE  and  place  in  the  IMG_LlNE’s  inatchlist  */ 

ilO. match  =  (MATCHTYPE  *)nialloc(sizeof(MATCHTYPE)))  ==  NULL) 
laiah"deiermine_malch:  creating  MATCHTYPE,  malloc\n”); 

match->line  =  m; 

match->angle_\  iew_diff  =  (m->est_anulc_to_image_eenier  -  i->angle_to_image_ccntcr); 
match->ne\i  =  NULL: 

’  Place  match  in  inatchlist  b\  increasing  order  of  angle_view_dil'f.  ’/ 


il  ti->matchlist  ==  NULL)  i->matchlist  =  match; 

else  ili  fah.Miiiaich->anglc_vicw_diff)  <  fabsti->matchlist->angle_view_difO) 

maich->ne.\l  =  i->maich!ist: 
i-:>matchlisi  =  match: 


place  =  i->matchlist; 

whileuplace->ncxt  !=  NULL)  &&  (,fabs(match->angle_view_difO  > 

fabs(place->next->angle_view_diff))) 


place  =  place->ncxt: 

I 

match->ne.xi  =  placc->next; 
place->next  =  match: 


)  /’  end  il\delta  ..) '  ; 
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/* . "7 

/*  void  find_vertical_matches(IMG_LINE  ’irngline,  LINE  ’modeMnc) 

/* 

/*  Function  to  find  all  possible  vertical  model  LINEs  that  could  be  matched 
/*  to  a  vertical  1MGJL1NE. 

r . V 

find_vertical_niaiches(imgline,  modelline) 

IMG_LINE  *imgline; 

LINE  ’■modelline; 

( 


while  (modelline  !=  NULL) 

i 

determine_veri_malch(modellinc,imgline): 
modelline  =  modellme->NEXT: 


imgline->pm  =  imgline->maichlist; 


r . */ 

/*  void  prins  matehlist', I\1G_LINE  *1) 

/* 

/’  Function  to  output  the  model  LINEs  matched  to  an  IMG_L1NE. 

/* . */ 

print_malchlisii  I ) 

1\1G_L1NE  ’1: 


MATCHTVPL  ’ m  =  i->malchlist: 
wlnleon  ’■=  MLL- 

piinth"  U  m  l<  .4lV'.  m->line->name,  m->anglc_vie\\_dil'D; 
m  =  m->ne\t; 


printl\"\n''n"i; 

) 

/’ . */ 

/*  void  dcclarcjnaichcM ,IMG_LINE  *1, 1MG_LINE  *m,  1MG_LINE  "r, 
/*  LINE  *modellines) 

/* 

/*  Declare  all  matches  to  model  LINEs  for  left,  middle,  and  right 
/*  vertical  1MG_L1NEs. 

/* . */ 

dec  lare_maiches(.l,m.r,  model  lines) 

I.MG_LINE  *1.  *m,  *r; 

LINE  ’inodellmes; 

( 

lind_vertical_maichestl,modellines); 

find_vcrtical_maiches(m,modcllines); 

find_veriical_matches(r,modellines); 

printf('VtImagc  line  matchlists  to  Model  line  NAMEfangle  difference): 
printfC'Ieft  ledge  Gi  >  \n’',l->name);  priiu_matchlist(l); 
printfp'middle  (edge  c.<  s)  > \n",m->namcV,  prinijnalchlisltm); 
printlCright  (edge  ci  si  >  \n".r->namc);  print_matchlisi(n; 


■\n”); 
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/* . */ 

/*  ini  detcrmine_posiu)ni(LINE  *1,  LINE  *m,  LINE  *r,  double  alpha  1,  double  alpha2, 
/*  POSE  *p,  double  *x,  double  *y) 

/* 

/*  Routine  to  determine  correct  position  based  upon  3  matched  model  LINEs  and 
/*  the  measured  angle  differences  from  the  IMG_LlNEs.  Returns  1  if  x,y 
/*  position  is  determined,  0  otherwise. 

/* . V 

int  determine_posilion(,l,  m,  r,  alpha  1,  alpha2,  p,  x,  y) 

LINE  *1,  *m,  *r;  /*  left,  middle,  and  right  viewed  model  LINEs  */ 
double  alpha  1,  alpha2;  /*  measured  image  angles  */ 

POSE  *p;  I*  estimated  (input)  POSE  */ 
double  *.\,  *\;  /*  x,y  location  to  be  returned  */ 

1 

double  chord  1  Jenglh,  chord2_lcngth, 

chordl_mid_x,  chordl_mid_y,  chord2_ntid_x,  chord2_mid_y, 

chord  1  _orient,  chord2_oricnt, 

hl_onent,  h2_orient,  hi,  h2, 

pvl_.\,  pvl_y,  pv2_x,  pv2_y, 

circle  I  .radius,  circlc2_radius, 

circle  1_\,  circle l_y,  circle2_x,  circlc2_y, 

c2c  1 .  c2c  1  _orient,  c2robot_orient,  c2_omega: 

/*  Ensure  that  3  different  model  LINEs  are  begin  used.  */ 

i f ( ( 1  ==  in)  .1  tm  ==  i  !  1 1  ==  r )) 

( 

t 

primlf'NO  posaion:  2  IMG_LINEs  matched  to  same  model  LlNE\n”>; 
return  0: 


/*  Determine  chord  lengths,  midpoints,  and  orientations.  */ 
chord  l_leneth  = 

sqrtuil->MODEL_X  -  m->MODEL_X  )*(1->M0DEL_X  -  m->MODEL_X  ))  + 
1 1  !->MODEL_  V  -  m->MODEL_Y  )*(l->MODEL_Y  -  m->MODEL_Y  ))); 


ehord2  Jeneih  = 

s\|iimiii.->MODEL_X  -  r->MODEL_X  j*(m->MODEL  X  -  r->MODEL_X  ))  + 
um->MODEL_Y  -  r->MODEL_Y  )*(m->MODEL_Y  -  r->MODEL_Y  ))); 

il\tchordl_lenelh  <-  Ml\_CHORD_LENGTHj  1!  (chord2_leneth  <  MIN_CHORD_LENGTH)) 

I 

primlf'NO  position:  chord  length  <  MIN_CHORD_LENGTH\n”); 
return  0: 

) 


chord l_mid_x  =  d->MODEL_X  +  m->MODEL_X  )/2.U: 
chord l_mid_\  =  t,l->MODEL_Y  +  m->MODEL_Y  )/2.0; 
chord2_mid_x  =  (m->MODEL_X  +  r->N10DEL_X  )/2.Q; 
chord2_mid_\  =  tm->MODEL_Y  +  r->MODEL_Y  )/2.0; 

chord i_oriem  =  map_aiieletaian2(l->MODEL_Y  -  m->MODEL  Y, 

I->MODEL_X  -  m->MODEL_X)i; 
chord2_orient  =  niap_aiiglei.aian2i.m->MODEL_Y  -  r->MODEL_Y, 

m->MODEL_X  -  r->MODEL_X)); 

/’  Determine  orientations  perpendicular  to  the  circle  chords  towards  input  pose.  */ 
hl_onent  =  normali/.et,chordl_orient  +  (PI/2.0) k 
h2_orient  =  normali/eichord2_oricnt  +  (PI/2.0)  >: 
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/*  Determine  perpendicular  lengths  from  the  center  of  the  chords  to  a  point  on  the  viewing  circle.  */ 
hi  =  (chord  l_lengih/2.0)  /  ian(alpha  1/2.0); 
h2  =  (chord)_lengih/2.0)  /  ian(alpha2/2.0); 

/*  Determine  the  viewing  points  perpendicular  to  the  chords.  */ 

pvl_x  =  chord l_mid_x  -  (hi  *  sin(hl_orient)); 

pvl_y  =  chordl_mid_y  +  (hi  *  cos(hl  .orient)); 

pv2_x  =  chord2_mid_x  -  (h2  *  sin(h2_oricnt)); 

pv2_y  =  chord2_mid_y  +  (h2  *  cos(h2_orient)); 

/*  Determine  radii  and  centers  of  viewing  circles.  */ 

circle  l_radius  =  sqrt((hl*hl)  +  (chordl  _lcngth*chordl_lenglh/4.0))/2.0; 

circlc2_radius  =  sqrt((h)*h))  +  (chord2_lcngth*chord2_lengih/4.0))/2.0; 

circle  1  _x  =  pvl.x  -  (circle l_radius  *  sin(hl_orient  +  PI)); 
circle  1  _y  =  pvl_y  +  (circle l_radius  *  cos(hl_oricnt  +  PI)); 
ctrcle2_x  =  pv2_x  -  (circle2_radius  *  sin(h2_orient  +  PI)); 
circle2_y  =  p\  2_y  +  (circle2_radius  *  cos(h2_oricnt  +  PI)); 

/*  Determine  distance  and  orientation  between  centers  of  viewing  circles.  */ 
c2el  =  sqm  ((circle  l_x  -  circle2_x)*(cirelel_x  -  circle2_x))  + 

(icirde  l_y  -  circlc2_y)*(circlel_y  -  circle2_y))); 

il'(c2c  1  ==  U.Oj 

I 

primlf'NO  poMiion:  viewpoint  circles  identicalW): 
reiurn  0: 


c2cl_orient  =  map_angle(aun2(cirdcl_\  -  cirde2_y,  circle  1  _.x  -  cirdc2_x)); 
c2robot_orieni  -  map_anglc(atan2(p->y  -  cirde2_y.  p->.\  -  circle2_x  >); 


I'  Calculate  \  lowing  poMtion.  ’/ 

if(circle2_radiu>  <  labs(c)c  1  -  circle l_radius))  c2_omega  =  0.U: 
else  if(cirde2_radius  >  ic2c  1  +  circle! .radius))  c2_omega  =  PI; 
else  c2_omcga  =  acoMi,ic2c  1  *  c2cl)  +  (circlc2_radius  *  eirclc2_radius)  - 

tcirde  1  .radius  *  circle  l.radius))  /  (2  *  c2cl  *  circlc2_radius)); 

il(lefi_ol(c2robot_oricnt.  dd  .orient)) 

( 

'  x  =  circlc2_.\  •  icirclc2_radius  *  sin(c2c  1  _orient+c2_omega ) ); 

=  circle)  \  -  tcirde2_radius  *  cos(c2cl_orieni+c2_omeca)); 

} 

else 

( 

•  \  =  circlc2_\  -  tcircle2_radius  *  sin(c2cl_orient-c2_omega)); 

=  circle)..)  ->■  icircle2_radius  *  cos(c)cl.orieni-c).omega)); 


reiurn  I ; 
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/* . */ 

/*  void  sclcct_ne.\t_best_maich  (MATCHTYPE  *1,  MATCHTYPE  *m, 

/*  MATCHTYPE  *r,  ini  *k) 

/* 

/*  Select  next  best  model  LINE  (minimum  fabs(angle_view_dil'0)  for  new 
/*  combination  in  the  next  detcrmine_position()  attempt. 

/* .  . V 

sclcci_next_besi_match(l,m,r,k) 

MATCHTYPE  *1,  *m,  *r;  /*  MATCHTYPE  pointers  for  left,  middle  and  right 
IMGJJNES  */ 

ini  *k;  /*  number  of  possible  pm->next  !=  NULL  */ 

i 

switch(*k) 

( 

case  3: 

il((labs(l->nexi->anglc_view_diff)  <  fabs(m->next->angle_view_diff)) 

&&  (fabs(l->ne.\l->anglc_view  diff)  <  l'abs(r->nexi->anglc_view_difO)) 

*k=  1; 

else  if((fabs(m->nexi->angle_vicw_diff)  <  fabs(l->next->angle_view_difO) 

(fabsim->nexi->angle_vicw_diff)  <  fabs(r->next->angle_view_diff))) 

*k  =  2: 
else  *k  =  3: 
break: 

case  2: 

il\l->ne\l  ==  NULL) 

i 

il\fabsim->nexi->angle_view_diff)  <  fabs(r->ncxt->anglc_view_difO)  *k  =  2; 
eKe  *k  =  3: 

) 

else  ib m->ne\t  ==  NULL) 

ififab.sO->ne\t->angle_vie\\_dilT)  <  labs(r->next->angle_view_diff))  *k  =  1; 

eKe  ’k  =  3: 

else’ 

ib  fabs(l->ne\t->angle_vicw_difO  <  fabstm->nexi->anele_vicw_diff))  *k  =  1; 
else  *k  =  2: 

break: 

case  1 : 

iltl->next  !=  NULL)  *k  =  1; 
else  ilim->ne\i  !=  NULL)  *k  =  2; 
else  "k  =  3: 
break: 

default: 

*k  =  0: 
break: 

)  /*  end  sv.  itch  *.• 

) 


118 


I* . •/ 

/*  double  dctcrmine_roiaiion  (LINE  *1,  LINE  *m,  LINE  *r,  double  x,  double  y, 

/*  double  intheia,  double  loffang,  double  moffang, 

/*  double  roffang) 

r 


/*  Function  that  determines  the  average  amount  of  rotational  correction 
/*  that  is  required  based  upon  the  3  model  LINEs  angular  offset  from  the 
/*  IMG_LlNEs  to  which  they  are  matched  with. 

/*  *! 


double  dctcrmine_rotaiion(l,  m,  r,  x,  y,  intheta,  loffang,  moffang,  roffang) 
LINE  *1,  *m,  *r; 

double  \,  y,  intheia,  loffang,  moffang,  roffang; 


/*  calculate  orientations  of  new  position  to  Vertical  Model  LINEs  */ 

double  lncworient  =  map_angle(atan2(l->MODEL_Y  -  y,  1->M0DEL_X  -  x)), 
mneworient  =  map_angle(atan2(m->MODEL_Y  -  y,  m->MODEL_X  -  x)), 
rneworient  =  map_angle(atan2(r->MODEL_Y  -  y,  r->MODEL_X  -  \)), 


/*  calculate  angle'  Horn  center  of  image  lor  new  position  */ 

lncwestangle  =  normalize(lnew orient  -  intheia), 
mncwcstangle  =  normali/.etmncworient  -  intheia ), 
rnewestangle  =  normali/ei  rneworient  -  intheia). 


/*  calculate  difference.'  \\  ah  offset  angles  from  image  V 

l_diff  =  normali/.eflnewestangle  -  loffang), 
m_diff  =  normali/etmneuestangk'  -  moffang), 
r_dill  =  normal i/.ei rnewestangle  •  roffang), 


/*  average  rotation  dillerence  */ 
rotation_dil  l  =  tl_difl  f  m_diff  +  r_di ff>  /  3.0; 

return  rotation_dil I : 
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/* . V 

/*  ini  verify_pose(double  x,  double  y,  double  theta,  IMG_LINE  *lmgLines, 

/*  LINE  *ModclLincs) 

/* 

/*  Function  to  return  number  of  IMGJJNEs  that  lie  over  model  LINEs 
I*  fora  corrected  2D  view  of  the  model  environment  fora  given  possible  pose. 

/* . . */ 

int  verify_posc(x,y,theia,ImgLines,ModelLines) 
double  x,y .theta; 

IMG_LINE  *ImgLines; 

LINE  ’’ModelLines; 

l 

LINE  *ml  =  ModelLines; 

IMG_L1NE  *il  =  ImgLines; 
double  della; 

int  cndpointjnclusion  =  0,  imgline_on_modclline  =  0,  nr_good_hits  =  0, 

found  =  0; 


/*  Calculate  esi_angle_to_image_cenier  for  all  ModelLines  based  upon 
pose;  (x,y, theta).  *7 
whilc(ml  !=  NLLL) 

( 

ml->esi_angle_to_image_center  = 

norntali/e(map_anule(atan2(ml->MODEL_Y-v,  ml->MODEL_X-x))  -  theta); 
ml  =  ml->NEXT: 


/*  Find  correspon<lenee>  of  image  lines  to  model  lines.  */ 
whilcfil  !=  NULL) 

I 

r  Ensure  \ertieal  image  line  is  not  due  to  image  borders.  */ 
il(.fabsul->angle_U)_image_cenicr)  <  F1ELD_HALF_ANGLE) 

j 

/*  Find  a  corresponding  1MG_L1NE.  */ 
ml  =  ModelLines; 
found  =  0; 

wlnleuml  !=  NULL)  Mound) 

) 

l 

della  =  (m)->Yl  -  il->pl  y)  +  (il->p2.y  -  ml->Y2); 

endpoint_indusion  = 

((delu  <=  0.0)  II 

(.(delta  >  0.0)  &.&.  (delta/il->d major  <  DELT A_LENGTH_R ATIO))); 

imgline_on_modelline  = 

(fabs(il->angle_to_image_centcr  - 

ml->csi_anglc_io_image_cemer)<VERlFY_EQ_ANGLE_EPSILON); 

iliendpoinijnclusion  &&  imgline_on_modclline) 

I 

++nr_good_hits; 
found  =  1 ; 


ml  =  ml->NEXT: 

l 

i 

I 

il  =  il->nv\l: 

I 

return  nr_good_tm.s; 

I 
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/* . V 

/*  POSE  *calculaie_bcst_pose(iIMG_LlNE  *1,  IMG_L1NE  *m,  IMG_LINE  *r, 

/*  double  alpha_lm,  double  alpha_mr,  POSE  *in_pose, 

I*  1MGJJNE  ’lmgLincs,  LINE  NModelLincs) 

/* 

I*  Function  that  conduce;  several  iterations  of  finding  an  1MG_LINE  to  LINE 
/*  triplet,  detennines  the  POSE  based  upon  the  matching,  and  returns  the  best  POSE 
/*  based  upon  number  of  IMGJJNE  to  model  LINE  matches  of  a  2D  view  of  the 
/*  best  POSE  and  translational  error  of  the  robot. 


/* 


V 


POSE  *caltulate_best_pose(l,m,r,alpha_lm,alpha_mr,in_pose,ImgLines,Mode!Lines) 
IMG_L1NE  *1,  *m,  *r,  *ImgLincs; 
double  alpha_lm,  alpha_inr; 

POSE  *in_pose; 

LINE  NModelLines: 


{ 

int 

POSE 

double 

MATCH  TV  PI. 


l  =  0,  k  =  0,  nr_hits  =  0,  best_nr_hits  =  0; 
*best_posc; 

\.  > ,  rot,  trans  =  0.0,  bcst_rot,  best_trans: 
*lmaich  =  l->matchlist, 

“mmalch  =  m->maichlisl, 

’rmatch  =  r->maich!ist: 


printip'Pose  determination: . \n”;; 

il((lvst_iX)se  =  i POSE  *  mialloctsi/eoffPOSE)))  ==  NULL) 
laiali"caleul:iie_besi_pose:  malloc\n”K 


ii((lmatch  ==  NULL.i  imniaich  ==  NULL)  ll(rmatch  ==  NULLo 

printl-  '  CTiii  not  determine  pose  for  ‘.i  >.  C  s.  G  m  NL  LL.  niaichlNlCn", 
1-  >n.ime.iii->naine.r->name): 


else  lory  =  0:  j  <  2(»:  --m  • 

1 

* 


prinili"  L‘ .  s,1  i  s.G  s|  ",lmaich->line->name.nimalch->lme->name, 
rmateh->line->name): 

ifuletermine_poMtion(lmatch->linc,  mmatch->line,  rmalch->line, 
alphajm,  alpha_mr,  in_pose,  &.\,  &y>) 


”  Calculate  translation  and  rotation  differences.  */ 

trails  =  ,M.|rt((in_pose->x  -  x)  *  (in_posc->.\  -  \)i  + 

((in_pose->y  -  y)  *  (in_pose->\  - 

rut  =  ileierimne_rotation(lmatch->line,  mmatch->line,  rmatch->line, 
x,  v,  in_posc->theta, 

!->angle_to„image_center, 

m->angle_to_imagc_center, 

r->angIc_lo_image_center); 

nrjiits  =  verify  _posc(x,y,in_pose->iheia  +  rot.ImgLines.ModclLincs); 
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printlf "\=  ck .21  y=  %.2f  T=  %.4f  R-  %.4f  nr  verify  maiches=  %d\n”, 
.\,y,irans,rot,nr_hils); 

ili(j  ==  (),i  II  (((doublc)nr_hils)/irans  >  ((double)besl_nr_hits)/best_trans)) 

I 

best_nr_hits  =  nr_hits;  /*  save  best  irans  and  roi  values  */ 
bcsi-trar.j  =  irans; 
bcst_roi  =  rot; 

bcsi_pose->x  =  x;  /*  assign  values  to  besi_posc  */ 
best_pose->y  =  y; 

best_pose->theta  =  in_pose->theia  +  rol; 

print! ("  ***  BEST  POSE  MODIFIED  ***\n”j; 

l->pm  =  Imatch;  /*  set  3  IMG_LINE’s  mateh  pointers  */ 

m->pm  =  mmalch; 

r->pm  =  rmatch; 


'*  Select  the  next  best  match  combination  of  model  LINEs.  */ 

k  =  t.t lmatch->ncxt  1=  NULL)  +  (mmatch->nc\l  !=  NULL)  + 
irmaich->nc\i  !=  NULL)): 

ilik  >  Hi 


.seleei_iieM_best_maich(lmaich,minaieh.rmateh.&k); 

il\  k  ==  1 )  lmaleh  =  lmateh->next; 

else  iltk  ==  2)  mmateh  =  mmaich->next; 

else  t ft  k  ==  3)  rmatch  =  rmatch->ne\t: 

I 

else  return  hcsijwse: 


return  Ivstjvsc. 

I 


/* . ’/ 

/*  POSE  *updatc_posc(lMG_LIN'E  ’Image-Lines,  LINE  ’ModcILines,  POSE  ’in_pose) 

/* 

/*  Function  to  return  a  corrected  POSE  given  the  vertical  edges  extracted  from  an  image 
/*  and  the  vertical  model  LINEs  that  should  be  visible  from  the  expected  POSE  of  the 
/*  robot. 

/*  . */ 

POSE  *updalc_posei.lmageLmes,ModcTLmes,in_pose) 

IMG_L1NE  ’ImageLines; 

LINE  ’.ModcILines: 

POSE  *in_pose: 

l 

IMG_L1NE  *il,  Melt,  ’middle,  ’right; 

POSE  ’out_pose; 

double  alphajm,  alpha_nir; 

ini  j; 


if( toui_po.se  =  (POSE  ’ imalloctsi/eoRPOSEf))  ==  NULL) 

lamb  "ui'datc. |\'nc.  creating  POSE  oul_posc:  malUx\n“): 


/*  Find  the  3  most  significant  I.MGJLINEs  and  return  them  in 
lell-to-righl  order  pointed  to  b>  ’left,  ’middle,  and  ’right.  ’/ 

il  =  ImageLines; 

primli, 'NnDcieimmc  3  major  image  edges  lor  matching: . \n  ): 


lory  =  1 ;  i  •  4;  i 

i 

H'.il  ==  \L  Li.. .  laiah "CAN  NOT  DETERMINE  POSE:  <  3  image  linesviT): 
’  Lii'Utc  \eitnal  image  line  is  not  due  to  image  borders.  */ 

1 1 1 labsi il->aiigle_to_image_ceiuer t  <  F1ELD  H  ALF_ ANGLE) 


ihj  ==  1 1 
loll  =  il: 

primlV  cid)  line  Gs,  angle  from  center  =  9<  .41\n",  j. 
il->name,  il->angle_to_image_center); 


else  ih ij  ==  2)  iiii  (fabs(il->anglc_lo_image_center  - 

left->angle_to_imagc_cemer)  >  IMG_LINE_MIN_ ANGLE)) 

ih  il->angle_to_imagc_ccnier  >  lelt->angle_to_image_center) 

right  =  left;  left  =  il: 

i 

else  right  =  il: 

printft"  9i d )  line  Gs.  angle  from  center  =  G  .41\n”.  j. 
il->name,  il->angle_lo_image_centen. 

-+j: 
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else  iU(J  ==3)  &.& 

Uabs(il->ang!e_to_image_center  - 

Icft->angle_lo_image_cenicr)  >  IMG_LINE_MIN_ANGLE)  &.&. 
i  labs(il->angle_io_image_cenier  - 

right->angle_to_image  center)  >  1MG'_LINE_MIN_ANGLE)) 

I 

il(il->angle_to_image_eenier  >  lelt->angle_to_imagc_center) 

i 

i 

middle  =  left;  left  =  il: 

) 

else  if\iI*>angle_io_irnage_ccnter  <  rmht->anule_to  iinage_eenter) 

I 

middle  =  richi;  rntht  =  •!: 

) 


else  middle  =  il; 

prmili"  ‘vd)  line  ^s,  angle  from  center  =  %.4l\n".  j, 
il->name,  iI->angle_lo_image_cemert. 


I 


il  =  il->ne\i: 


/*  Calculate  tew,  angles  kueen  the  three  major  IMG.LlNEs.  */ 

alpha_lm  =  leU->unglc_io_image_center  -  middlc->angle_u>_image_eenter; 
alpha_mr  =  middlc-Xmgh  t.  _image_ccnter  -  right->angle_lo_im.ige_cemer: 

/*  Declare  the  matches  to  model  LINEs  lor  the  three  major  IMG.LlNEs.  */ 
declare,  mate  hev  lei  t.  middle,  right.  Model  Lines;: 


r  Calculate  the  correct  pose.  ’/ 

out.pose  =  calculate_lvsi_poseilelt,  middle,  right,  alphajm,  alpha_mr, 

in_pose,  ImageLines,  ModelLincs); 


return  oul  J*ose: 
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C.  FILK:  MATCHDISI'LAY SLPPORT.H 


/’  FILENAME:  maichdisplaysupport.h 
/*  AUTHOR:  Kevin  Peterson 
/*  DATE:  07  February  1992 


** * *** 


/ 


/• 

/’DESCRIPTION:  Collection  ol  display  lunctions. 

/’ 

/*  void  displav_match_imaue  (NPSIMAGE  *img,  IMG_L1NE  *1, 

/*  '  “  LINE_HEAD  ’ml,  LINE_HEAD  ’m2) 

/’  void  draw_white_lmes(IMG_LINE  *1) 

/*  void  draw_white_model_Iines_with_names  (LINE  ’m) 

/’  void  draw_w  hiie_model_lines  (LINE  *m) 

/’  void  draw_bluck_model_hnes_wiih_names  (LINE  *m> 

/’  void  dravi_black_niodel_lines  (LINE  *m) 

/’  void  draw  _malch_  lines  (IMG_LINE  ’1) 

/’  void  display _match_loop  (NPSIMAGE  ’img,  IMG_LINE  *1, 

/’  L1NE_HEAD  ’ml,  ’L1NE_HEAD  ’m2,  long  winid) 

/’ 


/’  These  lunctions  make  calks  to  following  SiliconGraphics  routines: 
/’  preisize(i.  winopcin  i,  wmsett.),  wmcloseO,  RGBnuxle(), 

/*  singlebulleri  i,  gconligi  i,  «.|device(),  c3l\),  move2(),  draw2(), 

/*  swapbulTcrsi ),  Trectw  rue  >.  reshapes  iew  porttj.  cmo\2( ). 

/’  eharstri >,  and  clean 

.I************  »»Jfi*****»H«*J|f*****S*S»*.a»a*J»: 


/’ . ’/ 

/’  void  displavjnaich  imaee  i.NPSIMAGE  *ime,  IMG_L1\E  'I, 
/’  LINE_HEAD  "ml.  L1NE_HEAD  »m2i 


display  _maich_imageiinig.l.m  1  ,ni2y 

NPSIMAGE  'img:,/’  input  image  */ 

IMG.LINE  ”1:  *  list  of  IMG_LINEs  found  from  image  */ 

LINL_  HEAD  *  in  I .  m2;  /*  L1NE_HEAD  types  from  Jim's  "graphics. c"  */ 


long  u  umt: 


prcfsize(img->.\si/e,m.g->y  size);  /’  preferred  size  for  ssmdsiss  ’/ 
winid  =  ssinopenumg->name):  /*  open  the  window  */ 
RGBmodeO:  /’  set  RGBinode,  singlebufler,  and  */ 
singlebuffcro:  /’  configure  the  window  */ 
gconl  igi  >: 


/’  initiali/.e_conirols  ’ 
qdevicei  REDRAW); 
i|device(LEFTMOUSE  >. 
qdevice(MlDDLEMOUSEi: 
qdevicefRIGHTMOL'SEr 
qdevice(ESCKEY): 

display  _maich_lsis)pumg. I, m  I  ,m2,w  inid/: 

lrec(img->imgdaia.biisptr);  /*  delete  the  biunap  for  the  image  ’/ 
freefiingi:  /*  delete  the  NPSIMAGE  structure  */ 
wincloseiw misli:  /*  clsise  the  window  */ 
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7 


/* 

/*  void  draw_w  hite_hnestIMG_LlNE  *1) 

r 

I*  Draws  white  lines  over  an  image. 

/* . — -*/ 

draw_whiic_lines(l) 

IMGJLINE  *1: 

( 

sialic  float  wlmc|3]  =  !  1 .0,1. 0.1.0);  /*  rgb  w  hite  */ 

c3f(whiie); 

whilc(l!=NL'LL) 

( 

cmov2(l->pl  .x,l->pl  y);  charsLr(l->name); 
move2(l->pl ,\,l->pl.y);  draw2(l->p2.x,l->p2.yj; 
1  =  l->ne\t: 

) 

swapbullersi  >: 


r . */ 

r  void  draw  _w  Intc_modcl_lines_wiih_namcs  (LINE  "mi 
/* 

/*  Draws  blue  model  lines  over  an  nnaue. 

/* . r- . ■/ 

draw_w  hi ie_niodel_ lines  w  uh_namosim  t 
LINE  ’in: 

( 

sialic  lloal  \v hue:  3 1  =  ;  l  .0.1.0, 1 .0);  /*  rgb  while  */ 

c3fiw  hue): 
whilcim!=.M  LL  ■ 

1 

cinov 2i.ni->.\  1  .m->V  1 );  charsir(m->name); 
niove2iin->\  l.m->Y  1 1:  draw2i.m->X2,m->V2i: 
m  =  m->NL\T : 

I 

swapbullersi  i; 


/* . V 

/*  void  draw_black_model_imes_wiih_namcs  (LINE  *mi 
/* 

/*  Draw  s  black  model  lines  over  an  image. 

r  * / 

draw_black_nuvdel_lines  w  ith_names(m) 

LINE  *in: 

I 

static  lloal  blackl 3 i  =  ,u.o.(i.0.0.0);  /’  rgb  black  */ 

c3l'(black); 
w'hi)e(m!=NL  LL) 
i 

cinov 2(.m->\  1  ,m->Y  1 );  charstr(m->namc); 
move2(m->X  1  ,m->Y  1 ):  draw 2(m->X2,m->Y2); 
m  =  m->NEXT; 

) 

swapbullersi  y. 
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/ 


/* 

/*  void  draw  _w  hiie_model_lines  (LINE  *m) 

/* 

/*  Draws  while  model  lines  o\er  an  image. 

/* . */ 

draw_while_model_liiie.sim  > 

LINE  *m; 

{ 

static  float  whue|3]  =  i  1.0.1.0,1.01;  /*  rgb  while  */ 

c3l'(while): 

while(m!=NL'LL) 

( 

move2(m->X  l.m->\  1 );  draw2(m->X2,m->\ 2r. 
m  =  m->NEXT: 

) 

swapbullero  i; 


r . v 

/*  void  draw_hla«.h_iiK)del_lmes  (LINE  *inj 

r 

r  Draws  blaek  model  line-,  oier  ail  image. 

/* . . . L 

draw _blaek_nn>del  line v  no 
LINE  ‘"in: 

( 

suuie  lloal  blaek;  ,'i  =  ,  I'.i'.o.o.o.O!;  /”  rgb  blaek  ”7 

c3f(blaek  i; 
wliileim!  =  NLLL  > 

( 

mo\e2i  m->X  I  ,m->'i  1 1;  draw2un->X2.m->'i  2); 
m  =  m->NE\'I ; 

I 

swapbullerM  i; 
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/* . */ 

/*  void  display_match_loop  (NPSIMAGE  *img,  1MG_L1NE  *1, 

/*  LINE_HEAD  *ml,  L1NE.HEAD  *m2,  long  winid) 

/* . */ 

display_matchJoop(img,l,ml,m2,winid) 

NPSIMAGE  *img;  /*  input  image  */ 

IMG_LINE  *1;  /*  list  of  IMG_LINEs  found  from  image  */ 

LINE_HEAD  *ml,  *012;  /*  LINE_HEAD  type  from  Jim’s  “graphics.c”  */ 
long  winid; 

{ 

sialic  float  white! 3 J  =  1 1.0, 1.0, 1.0);  /*  rgb  white  */ 
short  value;  /*  value  returned  from  die  event  queue  */ 


/*  display  the  images  once  */ 
winsetf  winid); 

lrcctwriie(0,0,img->.\M/e  -  l,img->ysi/.e  -  l,img->imgdaia.bilspir); 
draw_white_lines(i); 


/*  loop  until  a  mouse  button  is  pressed  */ 
w'hilc(TRL'E) 


sw  i'chuiroadiM  value  n 

J 

ease  REDRAW : 

w  inset.,tlong)vaiae): 
reshapeviewporui: 
ili  \  alue  ==  w  mid) 


Ireeiwriiet0.0,img->.\si/e  -  l,img->>  si/e  -  1,  img->imgdata.bitspir); 
dtaw  _w  Intejines(l); 


break: 


ease  LEF-TMOLSE: 
ih  \ alue  ==  0i 

Ireetw  ntei 0,0.img->.\si/.e  -  l,img->ysi/.e  -  1,  img->imgdata.bitsptr); 
draw  _w  hue_model_lines_w  ith_namesfm  1  ->\'L1NE_L1ST); 
draw  _w  hitc_model_lmes(,m  1  ->L1NE_L1ST  >; 

/ 

break: 


ease  MIDDLE.MOl’SE: 
il  l  value  ==  O.i 

e3f(,w  lute;; 
clean,): 

draw  _hlack_model_lines_w  ith_namcs(m  1  ->VL1NE_L1ST); 
draw  black jnodeljincs' m  1->LINE_L!ST): 

break. 


ease  RIGHTMOUSE: 
if(  value  ==  0) 

I 

Ireetw  rite(0,0,img->xsize  -  l,img->ysize  -  1.  img->imgdata.biuspir); 
draw_white_modcl_lincs(m2->VLlNE_LIST ); 
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dra\\_whiie_model_lines(m2->LINE_LIST); 

) 

break: 

ease  ESCKtIY: 
return: 

default: 

break; 

j  /*  end  switch  ’./ 

)  I*  end  while  *, 
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APPENDIX  D-  USER’S  CLIDE 


file:  README 
date:  8  March  92 
by  :  Kevin  Peterson 

USER'S  MANUAL  FOR  YAMABICO' S  VISIUAL  NAVIGATION  ROUTINES 

This  file  outlines  the  functions  of  the  following  programs: 
findedge,  fastedge,  and  vertmatch 

I .  REQUIREMENT S : 

1.1  HARDWARE:  Tne  above  routines  were  developed  on  a 
SiiiccnGraphics  Personal  Iris  workstation  and  can  be  run  on 
ail  SGI  statists  at  NFS. 

1.2  SOFTWARE:  Ensure  the  following  files  are  in  the 
local  directory  f ; r  the  edge  extraction  routines: 

README  (tru;  : _ e  ) 

Makefile 
image_types  .  r. 
mat  ch_types . h 
npsimagesupp:  rt  .  r. 
edgesuppo  rt .  r. 
vert  suets  rt  .  n 
displays up p  .  .  . 

findedge . o 


2.1  Tne  f  :._1  .  :..r.u  C  program.  libraries  are  required: 
device . ;. 

ma  t  h . h 
st di  e  .  h 
gi  .  h 

gl /image..'. 

2.2  findedge,  sedge,  fastedge,  fastsobel,  vertedge, 

and  vertmatch  c=n  all  be  compiled  by  the  command  line  similar  to 
cc  -o  findedge  sir.dedge.c  -iimage  -lgl_s  -Irr. 
co  r astedcT  sastedge.c  -Iimage  -lgl_s  -lm 

etc.  .  . 

or  if  tr.e  Maxes ..e  ss  present: 
make  findedge 
make  fast- do-, 
etc  .  .  . 
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3.  PROGRAM  FUNCTIONS: 

3.4  FIN3EDGE  -  displays  the  grayscale  and  gradient 
images  of  the  input  image.  Conducts  edge  extraction  based 
upon  least  squares  fit. 

3.6  FASTED3E  -  displays  the  input  image  and  edges 
extracted.  Uses  the  green  component  of  the  RGB  pixel  value 
vice  the  grayscale  value  and  does  not  create  the  grayscale  or 
gradient  images. 

3.9  VERTKATCH  -  Extracts  vertical  edges  from  the  input 
image  (VERTEDGE) ,  creates  a  2D  view  of  the  model  environment 
based  upon  trie  estimated  pose  (dead-reckoning  maintained 
configuration)  of  the  robot,  conducts  line  matching  between 
image  vertical  edg^s  and  model  vertical  line  segments, 
determines  possible  poses  based  upon  different  matchings  for 
the  three  major  image  edges,  then  returns  the  best  pose  based 
upcr.  r.umbe r  ;f  edge  to  model  line  matches  and  distance  of 
updated  pose  t:  one  estimated  (input)  pose. 


4  . 


4.4  fir.dedge  «.filer.ame> 

Filename  is  input  picture  file  to  extract  edges  from.  Two 
windows  will  te  created;  one  for  the  grayscale  image,  the 
other  for  tr.e  gradient  image. 

-  Lines  extracted  oar.  be  drawn  onto  either  image  by  the 
LEFT  o:  MIDDLE  use  buttons  while  the  cursor  is 


^ r n e s  tor.  t  —  r r  a w r*  ever  a  w h o t e  t a c k g r  o u n d  by  MiDDLE 
mouse  butter.  while  tr.e  cursor  is  ir.  the  WINDOW. 

Exit:  r.1 1*HI  LEFT  mouse  buttons  (cursor  ir.  WINDOW)  or  escape  key. 


e.c  : as teoge  ^  e  i  _  e  r.  a  me  > 

One  window  rs  : p^r.eu  to  display  the  input  (oclor)  image  with 
extracted  edges  drawn  over  tr.e  image. 

-  Lines  extracted  oar.  be  drawn  onto  either  image  by  the 
LEFT  mouse  tut  tens  while  the  cursor  is  in  the  WINDOW. 

-  Lines  tar.  re  drawn  over  a  white  background  by  MIDDLE 
mouse  butt  on  while  the  cursor  is  in  the  WINDOW. 

-  Exit:  LEFT  buttons  (cursor  in  'WINDOW)  or  escape  key. 


4.9  vertmator.  <f iler.ame> 

One  window  is  opened  to  display  the  input  image  with 
the  extracted  vertical  edges. 

-  MIDDLE  rro-se  cutter,  (while  in  WINDOW)  displays  the  2D  wire-frame 
view  for  the  expected  pose  over  a  white  background. 

-  RIGHT  mouse  butt::,  displays  2D  expected  pose  wire-frame  view  over 

the 

input  picture. 

-  LEFT  mouse  butter,  displays  2D  wire-f ramie  view  from  the  corrected 

pose 

input  uicture. 


over  the 
-  Escape 
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